[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786361401922362-8267-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 11:30:02 dist-test-slave-srgv krb5kdc[8275](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 11:30:02 dist-test-slave-srgv krb5kdc[8275](info): set up 2 sockets
Aug 10 11:30:02 dist-test-slave-srgv krb5kdc[8275](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260810 11:30:03.228509 8267 mini_kdc.cc:121] Time spent starting KDC: real 1.100s user 0.007s sys 0.020s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786361401922362-8267-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786361401922362-8267-0/krb5kdc/test-user.keytab.
Aug 10 11:30:03 dist-test-slave-srgv krb5kdc[8275](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361403, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T11:30:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T11:30:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.18.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.18.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.18.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.254.keytab.
Entry for principal kudu/127.8.18.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.254.keytab.
I20260810 11:30:03.463088 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786361401922362-8267-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786361401922362-8267-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786361401922362-8267-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.18.254:38325
--webserver_interface=127.8.18.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.254.keytab
--principal=kudu/127.8.18.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.18.212:41199
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.18.254:38325
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786361401922362-8267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786361401922362-8267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786361401922362-8267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:30:04.158814 8288 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:30:04.159264 8288 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:30:04.159386 8288 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:30:04.168480 8288 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 11:30:04.168710 8288 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:30:04.168792 8288 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:30:04.168874 8288 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 11:30:04.168929 8288 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 11:30:04.181048 8288 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.18.212:41199
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786361401922362-8267-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786361401922362-8267-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.18.254:38325
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.254.keytab
--principal=kudu/127.8.18.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.18.254:38325
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.18.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786361401922362-8267-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:30:04.183171 8288 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:30:04.185717 8288 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:30:04.209777 8293 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:30:04.209918 8294 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:30:04.209723 8296 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:30:04.216005 8288 server_base.cc:1034] running on GCE node
I20260810 11:30:04.217340 8288 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:30:04.219905 8288 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:30:04.221293 8288 hybrid_clock.cc:648] HybridClock initialized: now 1786361404221187 us; error 112 us; skew 500 ppm
I20260810 11:30:04.221983 8288 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:30:04 dist-test-slave-srgv krb5kdc[8275](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361404, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:30:04.226413 8288 init.cc:377] Logged in from keytab as kudu/127.8.18.254@KRBTEST.COM (short username kudu)
I20260810 11:30:04.229317 8288 webserver.cc:460] Webserver started at http://127.8.18.254:38601/ using document root <none> and password file <none>
I20260810 11:30:04.230649 8288 fs_manager.cc:359] Metadata directory not provided
I20260810 11:30:04.230798 8288 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:30:04.231267 8288 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:30:04.236055 8288 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786361401922362-8267-0/minicluster-data/master-0/data/instance:
uuid: "aa242e8b6c084504badff521dcd9f736"
format_stamp: "Formatted at 2026-08-10 11:30:04 on dist-test-slave-srgv"
server_key: "dd6f2e0b76261ad899a1f59072e4bdbc"
server_key_iv: "540707b5bed1a74e4c6500a014403a68"
server_key_version: "encryptionkey@0"
I20260810 11:30:04.237020 8288 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786361401922362-8267-0/minicluster-data/master-0/wal/instance:
uuid: "aa242e8b6c084504badff521dcd9f736"
format_stamp: "Formatted at 2026-08-10 11:30:04 on dist-test-slave-srgv"
server_key: "dd6f2e0b76261ad899a1f59072e4bdbc"
server_key_iv: "540707b5bed1a74e4c6500a014403a68"
server_key_version: "encryptionkey@0"
I20260810 11:30:04.246675 8288 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.007s sys 0.000s
I20260810 11:30:04.254088 8303 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:30:04.256565 8288 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.001s sys 0.002s
I20260810 11:30:04.256973 8288 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786361401922362-8267-0/minicluster-data/master-0/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786361401922362-8267-0/minicluster-data/master-0/wal
uuid: "aa242e8b6c084504badff521dcd9f736"
format_stamp: "Formatted at 2026-08-10 11:30:04 on dist-test-slave-srgv"
server_key: "dd6f2e0b76261ad899a1f59072e4bdbc"
server_key_iv: "540707b5bed1a74e4c6500a014403a68"
server_key_version: "encryptionkey@0"
I20260810 11:30:04.257266 8288 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786361401922362-8267-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786361401922362-8267-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786361401922362-8267-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:30:04.296558 8288 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:30:04.303498 8288 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:30:04.303974 8288 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:30:04.335388 8288 rpc_server.cc:307] RPC server started. Bound to: 127.8.18.254:38325
I20260810 11:30:04.335434 8354 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.18.254:38325 every 8 connection(s)
I20260810 11:30:04.338104 8288 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
I20260810 11:30:04.346217 8355 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:30:04.348019 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 8288
I20260810 11:30:04.348608 8267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786361401922362-8267-0/minicluster-data/master-0/wal/instance
I20260810 11:30:04.352569 8267 external_mini_cluster.cc:1428] Setting key f74504215c0c30f2b38bdfba58ce9796
I20260810 11:30:04.368848 8355 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa242e8b6c084504badff521dcd9f736: Bootstrap starting.
I20260810 11:30:04.376147 8355 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P aa242e8b6c084504badff521dcd9f736: Neither blocks nor log segments found. Creating new log.
I20260810 11:30:04.379289 8355 log.cc:824] T 00000000000000000000000000000000 P aa242e8b6c084504badff521dcd9f736: Log is configured to *not* fsync() on all Append() calls
I20260810 11:30:04.384492 8355 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa242e8b6c084504badff521dcd9f736: No bootstrap required, opened a new log
I20260810 11:30:04.396807 8355 raft_consensus.cc:348] T 00000000000000000000000000000000 P aa242e8b6c084504badff521dcd9f736 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa242e8b6c084504badff521dcd9f736" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 38325 } }
I20260810 11:30:04.397496 8355 raft_consensus.cc:374] T 00000000000000000000000000000000 P aa242e8b6c084504badff521dcd9f736 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:30:04.397737 8355 raft_consensus.cc:729] T 00000000000000000000000000000000 P aa242e8b6c084504badff521dcd9f736 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa242e8b6c084504badff521dcd9f736, State: Initialized, Role: FOLLOWER
Aug 10 11:30:04 dist-test-slave-srgv krb5kdc[8275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361403, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.18.254@KRBTEST.COM
I20260810 11:30:04.399109 8355 consensus_queue.cc:260] T 00000000000000000000000000000000 P aa242e8b6c084504badff521dcd9f736 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa242e8b6c084504badff521dcd9f736" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 38325 } }
I20260810 11:30:04.399541 8355 raft_consensus.cc:388] T 00000000000000000000000000000000 P aa242e8b6c084504badff521dcd9f736 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 11:30:04.399847 8355 raft_consensus.cc:482] T 00000000000000000000000000000000 P aa242e8b6c084504badff521dcd9f736 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 11:30:04.400110 8355 raft_consensus.cc:3037] T 00000000000000000000000000000000 P aa242e8b6c084504badff521dcd9f736 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:30:04.404660 8355 raft_consensus.cc:504] T 00000000000000000000000000000000 P aa242e8b6c084504badff521dcd9f736 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa242e8b6c084504badff521dcd9f736" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 38325 } }
I20260810 11:30:04.406225 8355 leader_election.cc:302] T 00000000000000000000000000000000 P aa242e8b6c084504badff521dcd9f736 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: aa242e8b6c084504badff521dcd9f736; no voters:
I20260810 11:30:04.407970 8355 leader_election.cc:288] T 00000000000000000000000000000000 P aa242e8b6c084504badff521dcd9f736 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 11:30:04.408430 8360 raft_consensus.cc:2781] T 00000000000000000000000000000000 P aa242e8b6c084504badff521dcd9f736 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:30:04.411198 8360 raft_consensus.cc:686] T 00000000000000000000000000000000 P aa242e8b6c084504badff521dcd9f736 [term 1 LEADER]: Becoming Leader. State: Replica: aa242e8b6c084504badff521dcd9f736, State: Running, Role: LEADER
I20260810 11:30:04.412355 8355 sys_catalog.cc:564] T 00000000000000000000000000000000 P aa242e8b6c084504badff521dcd9f736 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 11:30:04.412470 8360 consensus_queue.cc:237] T 00000000000000000000000000000000 P aa242e8b6c084504badff521dcd9f736 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa242e8b6c084504badff521dcd9f736" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 38325 } }
I20260810 11:30:04.419417 8357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:04.360826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48288 (local address 127.8.18.254:38325)
0810 11:30:04.362637 (+ 1811us) server_negotiation.cc:206] Beginning negotiation
0810 11:30:04.362658 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0810 11:30:04.364656 (+ 1998us) server_negotiation.cc:407] Connection header received
0810 11:30:04.378147 (+ 13491us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:30:04.378188 (+ 41us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:30:04.379171 (+ 983us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:30:04.379878 (+ 707us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:30:04.384510 (+ 4632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:04.387516 (+ 3006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:04.390253 (+ 2737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:04.391834 (+ 1581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:04.392282 (+ 448us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:04.400876 (+ 8594us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:30:04.400968 (+ 92us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:30:04.401025 (+ 57us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:30:04.401117 (+ 92us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:30:04.406930 (+ 5813us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:04.409120 (+ 2190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:04.409144 (+ 24us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:04.409157 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:04.409646 (+ 489us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:04.410500 (+ 854us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:04.410531 (+ 31us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:04.410534 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:04.411864 (+ 1330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:30:04.412382 (+ 518us) server_negotiation.cc:1035] Waiting for connection context
0810 11:30:04.414454 (+ 2072us) server_negotiation.cc:299] Negotiation successful
0810 11:30:04.415576 (+ 1122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1188,"thread_start_us":657,"threads_started":1}
I20260810 11:30:04.422022 8361 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa242e8b6c084504badff521dcd9f736 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "aa242e8b6c084504badff521dcd9f736" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa242e8b6c084504badff521dcd9f736" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 38325 } } }
I20260810 11:30:04.422792 8361 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa242e8b6c084504badff521dcd9f736 [sys.catalog]: This master's current role is: LEADER
I20260810 11:30:04.432579 8362 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa242e8b6c084504badff521dcd9f736 [sys.catalog]: SysCatalogTable state changed. Reason: New leader aa242e8b6c084504badff521dcd9f736. Latest consensus state: current_term: 1 leader_uuid: "aa242e8b6c084504badff521dcd9f736" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa242e8b6c084504badff521dcd9f736" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 38325 } } }
I20260810 11:30:04.434137 8362 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa242e8b6c084504badff521dcd9f736 [sys.catalog]: This master's current role is: LEADER
I20260810 11:30:04.434885 8369 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 11:30:04.446676 8369 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 11:30:04.465081 8369 catalog_manager.cc:1347] Generated new cluster ID: b20330abf9e44af6b57ada1f800dc02e
I20260810 11:30:04.465289 8369 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 11:30:04.478005 8369 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 11:30:04.479653 8369 catalog_manager.cc:1504] Loading token signing keys...
I20260810 11:30:04.491868 8369 catalog_manager.cc:5868] T 00000000000000000000000000000000 P aa242e8b6c084504badff521dcd9f736: Generated new TSK 0
I20260810 11:30:04.493225 8369 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.18.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.18.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.18.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.193.keytab.
Entry for principal kudu/127.8.18.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.193.keytab.
I20260810 11:30:04.570335 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786361401922362-8267-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786361401922362-8267-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786361401922362-8267-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786361401922362-8267-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.18.193:0
--local_ip_for_outbound_sockets=127.8.18.193
--webserver_interface=127.8.18.193
--webserver_port=0
--tserver_master_addrs=127.8.18.254:38325
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.193.keytab
--principal=kudu/127.8.18.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.18.212:41199
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786361401922362-8267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786361401922362-8267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786361401922362-8267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:30:05.055182 8381 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:30:05.055697 8381 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:30:05.055907 8381 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:30:05.066974 8381 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:30:05.067209 8381 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:30:05.067466 8381 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.18.193
I20260810 11:30:05.079530 8381 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.18.212:41199
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786361401922362-8267-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786361401922362-8267-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.193.keytab
--principal=kudu/127.8.18.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.18.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786361401922362-8267-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.18.193
--webserver_port=0
--tserver_master_addrs=127.8.18.254:38325
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.18.193
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786361401922362-8267-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:30:05.081133 8381 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:30:05.082902 8381 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:30:05.102669 8387 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:30:05.103541 8389 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:30:05.105832 8386 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:30:05.107874 8381 server_base.cc:1034] running on GCE node
I20260810 11:30:05.109136 8381 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:30:05.110589 8381 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:30:05.111898 8381 hybrid_clock.cc:648] HybridClock initialized: now 1786361405111826 us; error 60 us; skew 500 ppm
I20260810 11:30:05.112417 8381 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:30:05 dist-test-slave-srgv krb5kdc[8275](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361405, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:30:05.116922 8381 init.cc:377] Logged in from keytab as kudu/127.8.18.193@KRBTEST.COM (short username kudu)
I20260810 11:30:05.119493 8381 webserver.cc:460] Webserver started at http://127.8.18.193:36595/ using document root <none> and password file <none>
I20260810 11:30:05.120385 8381 fs_manager.cc:359] Metadata directory not provided
I20260810 11:30:05.120518 8381 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:30:05.120941 8381 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:30:05.124825 8381 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786361401922362-8267-0/minicluster-data/ts-0/data/instance:
uuid: "38ccf01c41574ea2b75509302a1f3ef2"
format_stamp: "Formatted at 2026-08-10 11:30:05 on dist-test-slave-srgv"
server_key: "25f5c7dd6d712a1165fdf6d4d287249f"
server_key_iv: "1f581724ccf228172d6a549cb75e6580"
server_key_version: "encryptionkey@0"
I20260810 11:30:05.125877 8381 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786361401922362-8267-0/minicluster-data/ts-0/wal/instance:
uuid: "38ccf01c41574ea2b75509302a1f3ef2"
format_stamp: "Formatted at 2026-08-10 11:30:05 on dist-test-slave-srgv"
server_key: "25f5c7dd6d712a1165fdf6d4d287249f"
server_key_iv: "1f581724ccf228172d6a549cb75e6580"
server_key_version: "encryptionkey@0"
I20260810 11:30:05.133247 8381 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.004s sys 0.003s
I20260810 11:30:05.138695 8396 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:30:05.140616 8381 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260810 11:30:05.140877 8381 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786361401922362-8267-0/minicluster-data/ts-0/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786361401922362-8267-0/minicluster-data/ts-0/wal
uuid: "38ccf01c41574ea2b75509302a1f3ef2"
format_stamp: "Formatted at 2026-08-10 11:30:05 on dist-test-slave-srgv"
server_key: "25f5c7dd6d712a1165fdf6d4d287249f"
server_key_iv: "1f581724ccf228172d6a549cb75e6580"
server_key_version: "encryptionkey@0"
I20260810 11:30:05.141247 8381 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786361401922362-8267-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786361401922362-8267-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786361401922362-8267-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:30:05.166417 8381 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:30:05.171810 8381 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:30:05.172257 8381 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:30:05.173885 8381 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:30:05.176083 8381 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:30:05.176219 8381 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:30:05.176445 8381 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:30:05.176532 8381 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:30:05.242120 8381 rpc_server.cc:307] RPC server started. Bound to: 127.8.18.193:45699
I20260810 11:30:05.242173 8509 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.18.193:45699 every 8 connection(s)
I20260810 11:30:05.243695 8381 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786361401922362-8267-0/minicluster-data/ts-0/data/info.pb
I20260810 11:30:05.246816 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 8381
I20260810 11:30:05.247084 8267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786361401922362-8267-0/minicluster-data/ts-0/wal/instance
I20260810 11:30:05.248049 8267 external_mini_cluster.cc:1428] Setting key 0fdfedf7475b003b4fd7dcfef8ad0eb5
Aug 10 11:30:05 dist-test-slave-srgv krb5kdc[8275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361405, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.193@KRBTEST.COM for kudu/127.8.18.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.8.18.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.18.194@KRBTEST.COM" created.
I20260810 11:30:05.274048 8512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:05.248026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.193:48179 (local address 127.8.18.254:38325)
0810 11:30:05.248745 (+ 719us) server_negotiation.cc:206] Beginning negotiation
0810 11:30:05.248750 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:30:05.250141 (+ 1391us) server_negotiation.cc:407] Connection header received
0810 11:30:05.252452 (+ 2311us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:30:05.252460 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:30:05.252639 (+ 179us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:30:05.252820 (+ 181us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:30:05.254848 (+ 2028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:05.256299 (+ 1451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:05.259587 (+ 3288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:05.260092 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:05.260233 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:05.265652 (+ 5419us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:30:05.265659 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:30:05.265663 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:30:05.265707 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:30:05.270188 (+ 4481us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:05.271152 (+ 964us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:05.271158 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:05.271162 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:05.271343 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:05.271799 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:05.271803 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:05.271805 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:05.272170 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:30:05.272327 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0810 11:30:05.273286 (+ 959us) server_negotiation.cc:299] Negotiation successful
0810 11:30:05.273561 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":589,"thread_start_us":364,"threads_started":1}
I20260810 11:30:05.275203 8513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:05.248702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:38325 (local address 127.8.18.193:48179)
0810 11:30:05.249916 (+ 1214us) negotiation.cc:107] Waiting for socket to connect
0810 11:30:05.249968 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0810 11:30:05.252076 (+ 2108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:30:05.253144 (+ 1068us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:30:05.253155 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:30:05.253900 (+ 745us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:30:05.254614 (+ 714us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:30:05.254636 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:30:05.256599 (+ 1963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:30:05.256608 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:30:05.259362 (+ 2754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:30:05.259386 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:30:05.260450 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:30:05.260462 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:30:05.260833 (+ 371us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:05.261819 (+ 986us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:30:05.261857 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:30:05.265238 (+ 3381us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:30:05.270391 (+ 5153us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:30:05.270402 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:30:05.270429 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:30:05.270951 (+ 522us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:30:05.271464 (+ 513us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:30:05.271467 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:30:05.271470 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:30:05.271688 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:30:05.272328 (+ 640us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:30:05.272335 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:30:05.272724 (+ 389us) client_negotiation.cc:769] Sending connection context
0810 11:30:05.273144 (+ 420us) client_negotiation.cc:240] Negotiation successful
0810 11:30:05.273860 (+ 716us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":869,"thread_start_us":537,"threads_started":1}
I20260810 11:30:05.277127 8510 heartbeater.cc:347] Connected to a master server at 127.8.18.254:38325
I20260810 11:30:05.277681 8510 heartbeater.cc:464] Registering TS with master...
I20260810 11:30:05.278741 8510 heartbeater.cc:511] Master 127.8.18.254:38325 requested a full tablet report, sending...
I20260810 11:30:05.283430 8320 ts_manager.cc:194] Registered new tserver with Master: 38ccf01c41574ea2b75509302a1f3ef2 (127.8.18.193:45699)
I20260810 11:30:05.286972 8320 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.18.193@KRBTEST.COM'} at 127.8.18.193:48179
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.18.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.194.keytab.
Entry for principal kudu/127.8.18.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.194.keytab.
I20260810 11:30:05.306249 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786361401922362-8267-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786361401922362-8267-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786361401922362-8267-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786361401922362-8267-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.18.194:0
--local_ip_for_outbound_sockets=127.8.18.194
--webserver_interface=127.8.18.194
--webserver_port=0
--tserver_master_addrs=127.8.18.254:38325
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.194.keytab
--principal=kudu/127.8.18.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.18.212:41199
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786361401922362-8267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786361401922362-8267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786361401922362-8267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:30:05.766913 8516 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:30:05.767279 8516 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:30:05.767526 8516 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:30:05.776223 8516 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:30:05.776436 8516 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:30:05.776638 8516 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.18.194
I20260810 11:30:05.788388 8516 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.18.212:41199
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786361401922362-8267-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786361401922362-8267-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.194.keytab
--principal=kudu/127.8.18.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.18.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786361401922362-8267-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.18.194
--webserver_port=0
--tserver_master_addrs=127.8.18.254:38325
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.18.194
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786361401922362-8267-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:30:05.789983 8516 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:30:05.791488 8516 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:30:05.807476 8524 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:30:05.807744 8521 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:30:05.807745 8522 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:30:05.809070 8516 server_base.cc:1034] running on GCE node
I20260810 11:30:05.809923 8516 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:30:05.811044 8516 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:30:05.812351 8516 hybrid_clock.cc:648] HybridClock initialized: now 1786361405812272 us; error 70 us; skew 500 ppm
I20260810 11:30:05.812788 8516 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:30:05 dist-test-slave-srgv krb5kdc[8275](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361405, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:30:05.816808 8516 init.cc:377] Logged in from keytab as kudu/127.8.18.194@KRBTEST.COM (short username kudu)
I20260810 11:30:05.818799 8516 webserver.cc:460] Webserver started at http://127.8.18.194:42445/ using document root <none> and password file <none>
I20260810 11:30:05.819653 8516 fs_manager.cc:359] Metadata directory not provided
I20260810 11:30:05.819808 8516 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:30:05.820286 8516 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:30:05.823833 8516 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786361401922362-8267-0/minicluster-data/ts-1/data/instance:
uuid: "ecdbc0d8c1fe44c9a807688377a05ceb"
format_stamp: "Formatted at 2026-08-10 11:30:05 on dist-test-slave-srgv"
server_key: "f36ebbc62636bfdc45ba5fc4ede70bb2"
server_key_iv: "364c4c94c4cbb72a723fae109f10ec7a"
server_key_version: "encryptionkey@0"
I20260810 11:30:05.825003 8516 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786361401922362-8267-0/minicluster-data/ts-1/wal/instance:
uuid: "ecdbc0d8c1fe44c9a807688377a05ceb"
format_stamp: "Formatted at 2026-08-10 11:30:05 on dist-test-slave-srgv"
server_key: "f36ebbc62636bfdc45ba5fc4ede70bb2"
server_key_iv: "364c4c94c4cbb72a723fae109f10ec7a"
server_key_version: "encryptionkey@0"
I20260810 11:30:05.832432 8516 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.001s sys 0.005s
I20260810 11:30:05.836709 8531 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:30:05.838331 8516 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 11:30:05.838547 8516 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786361401922362-8267-0/minicluster-data/ts-1/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786361401922362-8267-0/minicluster-data/ts-1/wal
uuid: "ecdbc0d8c1fe44c9a807688377a05ceb"
format_stamp: "Formatted at 2026-08-10 11:30:05 on dist-test-slave-srgv"
server_key: "f36ebbc62636bfdc45ba5fc4ede70bb2"
server_key_iv: "364c4c94c4cbb72a723fae109f10ec7a"
server_key_version: "encryptionkey@0"
I20260810 11:30:05.838788 8516 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786361401922362-8267-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786361401922362-8267-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786361401922362-8267-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:30:05.870306 8516 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:30:05.876030 8516 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:30:05.876498 8516 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:30:05.877830 8516 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:30:05.879642 8516 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:30:05.879822 8516 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:30:05.879995 8516 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:30:05.880107 8516 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:30:05.932606 8516 rpc_server.cc:307] RPC server started. Bound to: 127.8.18.194:35109
I20260810 11:30:05.932718 8644 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.18.194:35109 every 8 connection(s)
I20260810 11:30:05.934095 8516 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786361401922362-8267-0/minicluster-data/ts-1/data/info.pb
I20260810 11:30:05.941589 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 8516
I20260810 11:30:05.941854 8267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786361401922362-8267-0/minicluster-data/ts-1/wal/instance
I20260810 11:30:05.942396 8267 external_mini_cluster.cc:1428] Setting key d94491ec0c1c95f66f9075eec7cd2198
Aug 10 11:30:05 dist-test-slave-srgv krb5kdc[8275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361405, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.194@KRBTEST.COM for kudu/127.8.18.254@KRBTEST.COM
I20260810 11:30:05.965986 8647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:05.937300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.194:60081 (local address 127.8.18.254:38325)
0810 11:30:05.937875 (+ 575us) server_negotiation.cc:206] Beginning negotiation
0810 11:30:05.937880 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:30:05.939260 (+ 1380us) server_negotiation.cc:407] Connection header received
0810 11:30:05.942081 (+ 2821us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:30:05.942088 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:30:05.942198 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:30:05.942333 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:30:05.944994 (+ 2661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:05.946828 (+ 1834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:05.950375 (+ 3547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:05.951326 (+ 951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:05.951538 (+ 212us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:05.956840 (+ 5302us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:30:05.956850 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:30:05.956858 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:30:05.956914 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:30:05.961533 (+ 4619us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:05.962643 (+ 1110us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:05.962652 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:05.962658 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:05.962903 (+ 245us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:05.963596 (+ 693us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:05.963602 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:05.963607 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:05.964061 (+ 454us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:30:05.964249 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0810 11:30:05.965203 (+ 954us) server_negotiation.cc:299] Negotiation successful
0810 11:30:05.965573 (+ 370us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":467,"thread_start_us":341,"threads_started":1}
WARNING: no policy specified for kudu/127.8.18.195@KRBTEST.COM; defaulting to no policy
I20260810 11:30:05.967208 8648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:05.937868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:38325 (local address 127.8.18.194:60081)
0810 11:30:05.939061 (+ 1193us) negotiation.cc:107] Waiting for socket to connect
0810 11:30:05.939108 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0810 11:30:05.941668 (+ 2560us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:30:05.942687 (+ 1019us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:30:05.942697 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:30:05.943571 (+ 874us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:30:05.944717 (+ 1146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:30:05.944753 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:30:05.947232 (+ 2479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:30:05.947242 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:30:05.950085 (+ 2843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:30:05.950114 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:30:05.951732 (+ 1618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:30:05.951744 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:30:05.952155 (+ 411us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:05.953084 (+ 929us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:30:05.953130 (+ 46us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:30:05.956473 (+ 3343us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:30:05.961779 (+ 5306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:30:05.961792 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:30:05.961817 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:30:05.962404 (+ 587us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:30:05.963147 (+ 743us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:30:05.963151 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:30:05.963154 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:30:05.963428 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:30:05.964243 (+ 815us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:30:05.964250 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:30:05.964681 (+ 431us) client_negotiation.cc:769] Sending connection context
0810 11:30:05.964941 (+ 260us) client_negotiation.cc:240] Negotiation successful
0810 11:30:05.965605 (+ 664us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":901,"thread_start_us":516,"threads_started":1}
I20260810 11:30:05.969096 8645 heartbeater.cc:347] Connected to a master server at 127.8.18.254:38325
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.18.195@KRBTEST.COM" created.
I20260810 11:30:05.969632 8645 heartbeater.cc:464] Registering TS with master...
I20260810 11:30:05.970551 8645 heartbeater.cc:511] Master 127.8.18.254:38325 requested a full tablet report, sending...
I20260810 11:30:05.972491 8320 ts_manager.cc:194] Registered new tserver with Master: ecdbc0d8c1fe44c9a807688377a05ceb (127.8.18.194:35109)
I20260810 11:30:05.974040 8320 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.18.194@KRBTEST.COM'} at 127.8.18.194:60081
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.18.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.195.keytab.
Entry for principal kudu/127.8.18.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.195.keytab.
I20260810 11:30:05.998276 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786361401922362-8267-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786361401922362-8267-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786361401922362-8267-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786361401922362-8267-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.18.195:0
--local_ip_for_outbound_sockets=127.8.18.195
--webserver_interface=127.8.18.195
--webserver_port=0
--tserver_master_addrs=127.8.18.254:38325
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.195.keytab
--principal=kudu/127.8.18.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.18.212:41199
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786361401922362-8267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786361401922362-8267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786361401922362-8267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 11:30:06.291855 8510 heartbeater.cc:503] Master 127.8.18.254:38325 was elected leader, sending a full tablet report...
W20260810 11:30:06.499583 8651 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:30:06.499977 8651 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:30:06.500178 8651 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:30:06.508520 8651 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:30:06.508754 8651 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:30:06.508955 8651 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.18.195
I20260810 11:30:06.520025 8651 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.18.212:41199
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786361401922362-8267-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786361401922362-8267-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.195.keytab
--principal=kudu/127.8.18.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.18.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786361401922362-8267-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.18.195
--webserver_port=0
--tserver_master_addrs=127.8.18.254:38325
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.18.195
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786361401922362-8267-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:30:06.521672 8651 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:30:06.523224 8651 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:30:06.538726 8656 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:30:06.538949 8657 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:30:06.538965 8659 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:30:06.540086 8651 server_base.cc:1034] running on GCE node
I20260810 11:30:06.540804 8651 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:30:06.542045 8651 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:30:06.543294 8651 hybrid_clock.cc:648] HybridClock initialized: now 1786361406543258 us; error 42 us; skew 500 ppm
I20260810 11:30:06.543675 8651 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:30:06 dist-test-slave-srgv krb5kdc[8275](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361406, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:30:06.547555 8651 init.cc:377] Logged in from keytab as kudu/127.8.18.195@KRBTEST.COM (short username kudu)
I20260810 11:30:06.549351 8651 webserver.cc:460] Webserver started at http://127.8.18.195:38953/ using document root <none> and password file <none>
I20260810 11:30:06.550181 8651 fs_manager.cc:359] Metadata directory not provided
I20260810 11:30:06.550352 8651 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:30:06.550709 8651 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:30:06.554327 8651 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786361401922362-8267-0/minicluster-data/ts-2/data/instance:
uuid: "2279d67752fe4c83aa477f1eea946814"
format_stamp: "Formatted at 2026-08-10 11:30:06 on dist-test-slave-srgv"
server_key: "97d6bb7631a1db9c00bae4f2b84fcb4e"
server_key_iv: "38da794753b4301c1c71b2b030fba6d8"
server_key_version: "encryptionkey@0"
I20260810 11:30:06.555395 8651 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786361401922362-8267-0/minicluster-data/ts-2/wal/instance:
uuid: "2279d67752fe4c83aa477f1eea946814"
format_stamp: "Formatted at 2026-08-10 11:30:06 on dist-test-slave-srgv"
server_key: "97d6bb7631a1db9c00bae4f2b84fcb4e"
server_key_iv: "38da794753b4301c1c71b2b030fba6d8"
server_key_version: "encryptionkey@0"
I20260810 11:30:06.562856 8651 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.001s
I20260810 11:30:06.567752 8666 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:30:06.570056 8651 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260810 11:30:06.570307 8651 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786361401922362-8267-0/minicluster-data/ts-2/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786361401922362-8267-0/minicluster-data/ts-2/wal
uuid: "2279d67752fe4c83aa477f1eea946814"
format_stamp: "Formatted at 2026-08-10 11:30:06 on dist-test-slave-srgv"
server_key: "97d6bb7631a1db9c00bae4f2b84fcb4e"
server_key_iv: "38da794753b4301c1c71b2b030fba6d8"
server_key_version: "encryptionkey@0"
I20260810 11:30:06.570565 8651 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786361401922362-8267-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786361401922362-8267-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786361401922362-8267-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:30:06.599566 8651 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:30:06.605077 8651 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:30:06.605490 8651 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:30:06.606688 8651 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:30:06.608495 8651 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:30:06.608671 8651 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:30:06.608937 8651 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:30:06.609076 8651 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:30:06.667958 8651 rpc_server.cc:307] RPC server started. Bound to: 127.8.18.195:42393
I20260810 11:30:06.668020 8778 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.18.195:42393 every 8 connection(s)
I20260810 11:30:06.669627 8651 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786361401922362-8267-0/minicluster-data/ts-2/data/info.pb
I20260810 11:30:06.674175 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 8651
I20260810 11:30:06.674430 8267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786361401922362-8267-0/minicluster-data/ts-2/wal/instance
I20260810 11:30:06.675100 8267 external_mini_cluster.cc:1428] Setting key bdfc915c1b8bf1b62a90ced89265e164
Aug 10 11:30:06 dist-test-slave-srgv krb5kdc[8275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361406, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.195@KRBTEST.COM for kudu/127.8.18.254@KRBTEST.COM
I20260810 11:30:06.697291 8781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:06.672859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.195:55173 (local address 127.8.18.254:38325)
0810 11:30:06.673629 (+ 770us) server_negotiation.cc:206] Beginning negotiation
0810 11:30:06.673637 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:30:06.674863 (+ 1226us) server_negotiation.cc:407] Connection header received
0810 11:30:06.678148 (+ 3285us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:30:06.678153 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:30:06.678268 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:30:06.678403 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:30:06.680982 (+ 2579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:06.681981 (+ 999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:06.684136 (+ 2155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:06.684744 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:06.684900 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:06.689591 (+ 4691us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:30:06.689597 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:30:06.689600 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:30:06.689641 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:30:06.693345 (+ 3704us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:06.694314 (+ 969us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:06.694320 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:06.694324 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:06.694568 (+ 244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:06.695095 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:06.695103 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:06.695107 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:06.695488 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:30:06.695671 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0810 11:30:06.696506 (+ 835us) server_negotiation.cc:299] Negotiation successful
0810 11:30:06.696787 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":642,"thread_start_us":479,"threads_started":1}
I20260810 11:30:06.698608 8782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:06.673476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:38325 (local address 127.8.18.195:55173)
0810 11:30:06.674649 (+ 1173us) negotiation.cc:107] Waiting for socket to connect
0810 11:30:06.674703 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0810 11:30:06.677779 (+ 3076us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:30:06.678776 (+ 997us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:30:06.678793 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:30:06.679927 (+ 1134us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:30:06.680774 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:30:06.680798 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:30:06.682219 (+ 1421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:30:06.682230 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:30:06.683930 (+ 1700us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:30:06.683945 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:30:06.685061 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:30:06.685071 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:30:06.685390 (+ 319us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:06.686284 (+ 894us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:30:06.686309 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:30:06.689154 (+ 2845us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:30:06.693583 (+ 4429us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:30:06.693595 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:30:06.693623 (+ 28us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:30:06.694160 (+ 537us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:30:06.694719 (+ 559us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:30:06.694725 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:30:06.694729 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:30:06.694962 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:30:06.695642 (+ 680us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:30:06.695652 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:30:06.696077 (+ 425us) client_negotiation.cc:769] Sending connection context
0810 11:30:06.696401 (+ 324us) client_negotiation.cc:240] Negotiation successful
0810 11:30:06.697047 (+ 646us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":810,"thread_start_us":517,"threads_started":1}
I20260810 11:30:06.700486 8779 heartbeater.cc:347] Connected to a master server at 127.8.18.254:38325
I20260810 11:30:06.700940 8779 heartbeater.cc:464] Registering TS with master...
I20260810 11:30:06.702188 8779 heartbeater.cc:511] Master 127.8.18.254:38325 requested a full tablet report, sending...
I20260810 11:30:06.704352 8320 ts_manager.cc:194] Registered new tserver with Master: 2279d67752fe4c83aa477f1eea946814 (127.8.18.195:42393)
I20260810 11:30:06.705564 8320 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.18.195@KRBTEST.COM'} at 127.8.18.195:55173
I20260810 11:30:06.707342 8267 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 10 11:30:06 dist-test-slave-srgv krb5kdc[8275](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361406, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 10 11:30:06 dist-test-slave-srgv krb5kdc[8275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361406, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.8.18.193@KRBTEST.COM
I20260810 11:30:06.769709 8787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:06.750610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51592 (local address 127.8.18.193:45699)
0810 11:30:06.751536 (+ 926us) server_negotiation.cc:206] Beginning negotiation
0810 11:30:06.751544 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:30:06.751577 (+ 33us) server_negotiation.cc:407] Connection header received
0810 11:30:06.751778 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:30:06.751787 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:30:06.752021 (+ 234us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:30:06.752324 (+ 303us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:30:06.754307 (+ 1983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:06.755403 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:06.757013 (+ 1610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:06.757749 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:06.757861 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:06.762706 (+ 4845us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:30:06.762723 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:30:06.762739 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:30:06.762781 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:30:06.766345 (+ 3564us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:06.767228 (+ 883us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:06.767237 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:06.767239 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:06.767392 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:06.767857 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:06.767860 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:06.767862 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:06.768254 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:30:06.768515 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0810 11:30:06.768859 (+ 344us) server_negotiation.cc:299] Negotiation successful
0810 11:30:06.769090 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":767,"thread_start_us":491,"threads_started":1}
W20260810 11:30:06.771526 8424 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51592
Aug 10 11:30:06 dist-test-slave-srgv krb5kdc[8275](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361406, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 10 11:30:06 dist-test-slave-srgv krb5kdc[8275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361406, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.18.193@KRBTEST.COM
I20260810 11:30:06.830618 8787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:06.812554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51608 (local address 127.8.18.193:45699)
0810 11:30:06.812809 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0810 11:30:06.812817 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:30:06.813164 (+ 347us) server_negotiation.cc:407] Connection header received
0810 11:30:06.813376 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:30:06.813388 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:30:06.813556 (+ 168us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:30:06.813675 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:30:06.815390 (+ 1715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:06.816187 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:06.817801 (+ 1614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:06.818360 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:06.818528 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:06.823978 (+ 5450us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:30:06.823984 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:30:06.823988 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:30:06.824084 (+ 96us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:30:06.827375 (+ 3291us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:06.828421 (+ 1046us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:06.828427 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:06.828429 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:06.828614 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:06.829141 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:06.829146 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:06.829148 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:06.829546 (+ 398us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:30:06.829778 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0810 11:30:06.830151 (+ 373us) server_negotiation.cc:299] Negotiation successful
0810 11:30:06.830352 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260810 11:30:06.832816 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 8381
I20260810 11:30:06.849807 8787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:06.834476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51620 (local address 127.8.18.193:45699)
0810 11:30:06.834770 (+ 294us) server_negotiation.cc:206] Beginning negotiation
0810 11:30:06.834776 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:30:06.834927 (+ 151us) server_negotiation.cc:407] Connection header received
0810 11:30:06.835196 (+ 269us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:30:06.835200 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:30:06.835263 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:30:06.835370 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:30:06.836729 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:06.837531 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:06.839064 (+ 1533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:06.839541 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:06.839685 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:06.842917 (+ 3232us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:30:06.842923 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:30:06.842926 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:30:06.842995 (+ 69us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:30:06.846532 (+ 3537us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:06.847469 (+ 937us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:06.847474 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:06.847476 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:06.847647 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:06.848198 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:06.848202 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:06.848204 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:06.848551 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:30:06.848777 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0810 11:30:06.849104 (+ 327us) server_negotiation.cc:299] Negotiation successful
0810 11:30:06.849289 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260810 11:30:06.850880 8505 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:30:06.941715 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 8381
I20260810 11:30:06.960614 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 8516
Aug 10 11:30:06 dist-test-slave-srgv krb5kdc[8275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361406, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.18.194@KRBTEST.COM
I20260810 11:30:06.978256 8645 heartbeater.cc:503] Master 127.8.18.254:38325 was elected leader, sending a full tablet report...
I20260810 11:30:06.979609 8794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:06.961735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41214 (local address 127.8.18.194:35109)
0810 11:30:06.962430 (+ 695us) server_negotiation.cc:206] Beginning negotiation
0810 11:30:06.962438 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:30:06.962465 (+ 27us) server_negotiation.cc:407] Connection header received
0810 11:30:06.962534 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:30:06.962540 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:30:06.962754 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:30:06.962979 (+ 225us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:30:06.964407 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:06.965567 (+ 1160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:06.966916 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:06.967585 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:06.967709 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:06.972375 (+ 4666us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:30:06.972391 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:30:06.972407 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:30:06.972458 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:30:06.976079 (+ 3621us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:06.977058 (+ 979us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:06.977067 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:06.977070 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:06.977275 (+ 205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:06.977800 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:06.977804 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:06.977806 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:06.978223 (+ 417us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:30:06.978627 (+ 404us) server_negotiation.cc:1035] Waiting for connection context
0810 11:30:06.978929 (+ 302us) server_negotiation.cc:299] Negotiation successful
0810 11:30:06.979223 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":531,"thread_start_us":370,"threads_started":1}
I20260810 11:30:06.980996 8640 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:30:07.066630 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 8516
I20260810 11:30:07.086639 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 8651
Aug 10 11:30:07 dist-test-slave-srgv krb5kdc[8275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361406, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.18.195@KRBTEST.COM
I20260810 11:30:07.105726 8796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:07.087946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44824 (local address 127.8.18.195:42393)
0810 11:30:07.088525 (+ 579us) server_negotiation.cc:206] Beginning negotiation
0810 11:30:07.088533 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:30:07.088565 (+ 32us) server_negotiation.cc:407] Connection header received
0810 11:30:07.088632 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:30:07.088638 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:30:07.088850 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:30:07.088984 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:30:07.090557 (+ 1573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:07.091691 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:07.093067 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:07.093864 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:07.093988 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:07.098750 (+ 4762us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:30:07.098769 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:30:07.098787 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:30:07.098834 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:30:07.102379 (+ 3545us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:07.103250 (+ 871us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:07.103259 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:07.103262 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:07.103414 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:07.103841 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:07.103845 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:07.103847 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:07.104269 (+ 422us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:30:07.104532 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0810 11:30:07.104924 (+ 392us) server_negotiation.cc:299] Negotiation successful
0810 11:30:07.105163 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":426,"thread_start_us":287,"threads_started":1}
I20260810 11:30:07.107323 8774 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:30:07.192890 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 8651
I20260810 11:30:07.210690 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 8288
I20260810 11:30:07.212059 8350 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:30:07.317188 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 8288
2026-08-10T11:30:07Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (5256 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786361401922362-8267-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 11:30:07 dist-test-slave-srgv krb5kdc[8801](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 11:30:07 dist-test-slave-srgv krb5kdc[8801](info): set up 2 sockets
Aug 10 11:30:07 dist-test-slave-srgv krb5kdc[8801](info): commencing operation
krb5kdc: starting...
W20260810 11:30:08.448590 8267 mini_kdc.cc:121] Time spent starting KDC: real 1.084s user 0.001s sys 0.018s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786361401922362-8267-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786361401922362-8267-0/krb5kdc/test-user.keytab.
Aug 10 11:30:08 dist-test-slave-srgv krb5kdc[8801](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361408, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T11:30:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T11:30:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.18.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.18.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.18.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.254.keytab.
Entry for principal kudu/127.8.18.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.254.keytab.
I20260810 11:30:08.670907 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786361401922362-8267-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786361401922362-8267-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786361401922362-8267-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.18.254:35605
--webserver_interface=127.8.18.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.254.keytab
--principal=kudu/127.8.18.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.18.212:35165
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.18.254:35605
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786361401922362-8267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786361401922362-8267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786361401922362-8267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:30:09.029963 8814 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:30:09.030345 8814 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:30:09.030447 8814 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:30:09.038574 8814 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 11:30:09.038733 8814 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:30:09.038784 8814 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:30:09.038836 8814 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 11:30:09.038873 8814 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 11:30:09.049710 8814 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.18.212:35165
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786361401922362-8267-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786361401922362-8267-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.18.254:35605
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.254.keytab
--principal=kudu/127.8.18.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.18.254:35605
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.18.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786361401922362-8267-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:30:09.051227 8814 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:30:09.052760 8814 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:30:09.067039 8819 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:30:09.066982 8822 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:30:09.067006 8820 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:30:09.067766 8814 server_base.cc:1034] running on GCE node
I20260810 11:30:09.068833 8814 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:30:09.070740 8814 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:30:09.072043 8814 hybrid_clock.cc:648] HybridClock initialized: now 1786361409071971 us; error 74 us; skew 500 ppm
I20260810 11:30:09.072448 8814 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:30:09 dist-test-slave-srgv krb5kdc[8801](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361409, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:30:09.077296 8814 init.cc:377] Logged in from keytab as kudu/127.8.18.254@KRBTEST.COM (short username kudu)
I20260810 11:30:09.079561 8814 webserver.cc:460] Webserver started at http://127.8.18.254:38863/ using document root <none> and password file <none>
I20260810 11:30:09.080421 8814 fs_manager.cc:359] Metadata directory not provided
I20260810 11:30:09.080554 8814 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:30:09.080873 8814 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:30:09.084673 8814 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786361401922362-8267-0/minicluster-data/master-0/data/instance:
uuid: "f9409add55f141f2b144df70f51c2bbe"
format_stamp: "Formatted at 2026-08-10 11:30:09 on dist-test-slave-srgv"
server_key: "64b58a42cbd4f60974281f61c5dd9b3d"
server_key_iv: "e70273ed6f2f045a161a127d87a67660"
server_key_version: "encryptionkey@0"
I20260810 11:30:09.085817 8814 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786361401922362-8267-0/minicluster-data/master-0/wal/instance:
uuid: "f9409add55f141f2b144df70f51c2bbe"
format_stamp: "Formatted at 2026-08-10 11:30:09 on dist-test-slave-srgv"
server_key: "64b58a42cbd4f60974281f61c5dd9b3d"
server_key_iv: "e70273ed6f2f045a161a127d87a67660"
server_key_version: "encryptionkey@0"
I20260810 11:30:09.092880 8814 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.000s
I20260810 11:30:09.097868 8829 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:30:09.100008 8814 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260810 11:30:09.100427 8814 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786361401922362-8267-0/minicluster-data/master-0/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786361401922362-8267-0/minicluster-data/master-0/wal
uuid: "f9409add55f141f2b144df70f51c2bbe"
format_stamp: "Formatted at 2026-08-10 11:30:09 on dist-test-slave-srgv"
server_key: "64b58a42cbd4f60974281f61c5dd9b3d"
server_key_iv: "e70273ed6f2f045a161a127d87a67660"
server_key_version: "encryptionkey@0"
I20260810 11:30:09.100776 8814 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786361401922362-8267-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786361401922362-8267-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786361401922362-8267-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:30:09.131496 8814 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:30:09.136917 8814 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:30:09.137295 8814 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:30:09.159005 8814 rpc_server.cc:307] RPC server started. Bound to: 127.8.18.254:35605
I20260810 11:30:09.159045 8880 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.18.254:35605 every 8 connection(s)
I20260810 11:30:09.160627 8814 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
I20260810 11:30:09.164664 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 8814
I20260810 11:30:09.164945 8267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786361401922362-8267-0/minicluster-data/master-0/wal/instance
I20260810 11:30:09.165356 8881 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:30:09.165858 8267 external_mini_cluster.cc:1428] Setting key 4e9fa068e1fedc235e02354beff7b117
I20260810 11:30:09.182587 8881 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f9409add55f141f2b144df70f51c2bbe: Bootstrap starting.
Aug 10 11:30:09 dist-test-slave-srgv krb5kdc[8801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361408, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.18.254@KRBTEST.COM
I20260810 11:30:09.187438 8881 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f9409add55f141f2b144df70f51c2bbe: Neither blocks nor log segments found. Creating new log.
I20260810 11:30:09.189033 8881 log.cc:824] T 00000000000000000000000000000000 P f9409add55f141f2b144df70f51c2bbe: Log is configured to *not* fsync() on all Append() calls
I20260810 11:30:09.193205 8881 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f9409add55f141f2b144df70f51c2bbe: No bootstrap required, opened a new log
I20260810 11:30:09.194594 8884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:09.168182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48658 (local address 127.8.18.254:35605)
0810 11:30:09.169111 (+ 929us) server_negotiation.cc:206] Beginning negotiation
0810 11:30:09.169124 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 11:30:09.169171 (+ 47us) server_negotiation.cc:407] Connection header received
0810 11:30:09.171029 (+ 1858us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:30:09.171082 (+ 53us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:30:09.171582 (+ 500us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:30:09.171983 (+ 401us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:30:09.174208 (+ 2225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:09.175645 (+ 1437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:09.177701 (+ 2056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:09.178554 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:09.178740 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:09.184195 (+ 5455us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:30:09.184211 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:30:09.184232 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:30:09.184300 (+ 68us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:30:09.188874 (+ 4574us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:09.190166 (+ 1292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:09.190175 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:09.190183 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:09.190414 (+ 231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:09.190966 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:09.190971 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:09.190974 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:09.191733 (+ 759us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:30:09.192061 (+ 328us) server_negotiation.cc:1035] Waiting for connection context
0810 11:30:09.192575 (+ 514us) server_negotiation.cc:299] Negotiation successful
0810 11:30:09.193081 (+ 506us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":677,"thread_start_us":431,"threads_started":1}
I20260810 11:30:09.202647 8881 raft_consensus.cc:348] T 00000000000000000000000000000000 P f9409add55f141f2b144df70f51c2bbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9409add55f141f2b144df70f51c2bbe" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 35605 } }
I20260810 11:30:09.203065 8881 raft_consensus.cc:374] T 00000000000000000000000000000000 P f9409add55f141f2b144df70f51c2bbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:30:09.203148 8881 raft_consensus.cc:729] T 00000000000000000000000000000000 P f9409add55f141f2b144df70f51c2bbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9409add55f141f2b144df70f51c2bbe, State: Initialized, Role: FOLLOWER
I20260810 11:30:09.203768 8881 consensus_queue.cc:260] T 00000000000000000000000000000000 P f9409add55f141f2b144df70f51c2bbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9409add55f141f2b144df70f51c2bbe" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 35605 } }
I20260810 11:30:09.204027 8881 raft_consensus.cc:388] T 00000000000000000000000000000000 P f9409add55f141f2b144df70f51c2bbe [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 11:30:09.204144 8881 raft_consensus.cc:482] T 00000000000000000000000000000000 P f9409add55f141f2b144df70f51c2bbe [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 11:30:09.204288 8881 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f9409add55f141f2b144df70f51c2bbe [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:30:09.207048 8881 raft_consensus.cc:504] T 00000000000000000000000000000000 P f9409add55f141f2b144df70f51c2bbe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9409add55f141f2b144df70f51c2bbe" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 35605 } }
I20260810 11:30:09.207811 8881 leader_election.cc:302] T 00000000000000000000000000000000 P f9409add55f141f2b144df70f51c2bbe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f9409add55f141f2b144df70f51c2bbe; no voters:
I20260810 11:30:09.208462 8881 leader_election.cc:288] T 00000000000000000000000000000000 P f9409add55f141f2b144df70f51c2bbe [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 11:30:09.208632 8886 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f9409add55f141f2b144df70f51c2bbe [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:30:09.209388 8886 raft_consensus.cc:686] T 00000000000000000000000000000000 P f9409add55f141f2b144df70f51c2bbe [term 1 LEADER]: Becoming Leader. State: Replica: f9409add55f141f2b144df70f51c2bbe, State: Running, Role: LEADER
I20260810 11:30:09.210114 8886 consensus_queue.cc:237] T 00000000000000000000000000000000 P f9409add55f141f2b144df70f51c2bbe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9409add55f141f2b144df70f51c2bbe" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 35605 } }
I20260810 11:30:09.210851 8881 sys_catalog.cc:564] T 00000000000000000000000000000000 P f9409add55f141f2b144df70f51c2bbe [sys.catalog]: configured and running, proceeding with master startup.
I20260810 11:30:09.212968 8887 sys_catalog.cc:455] T 00000000000000000000000000000000 P f9409add55f141f2b144df70f51c2bbe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f9409add55f141f2b144df70f51c2bbe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9409add55f141f2b144df70f51c2bbe" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 35605 } } }
I20260810 11:30:09.213492 8887 sys_catalog.cc:458] T 00000000000000000000000000000000 P f9409add55f141f2b144df70f51c2bbe [sys.catalog]: This master's current role is: LEADER
I20260810 11:30:09.215626 8888 sys_catalog.cc:455] T 00000000000000000000000000000000 P f9409add55f141f2b144df70f51c2bbe [sys.catalog]: SysCatalogTable state changed. Reason: New leader f9409add55f141f2b144df70f51c2bbe. Latest consensus state: current_term: 1 leader_uuid: "f9409add55f141f2b144df70f51c2bbe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9409add55f141f2b144df70f51c2bbe" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 35605 } } }
I20260810 11:30:09.216128 8888 sys_catalog.cc:458] T 00000000000000000000000000000000 P f9409add55f141f2b144df70f51c2bbe [sys.catalog]: This master's current role is: LEADER
I20260810 11:30:09.219108 8892 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 11:30:09.225827 8892 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 11:30:09.236330 8892 catalog_manager.cc:1347] Generated new cluster ID: 90eb3b587b584e54837ca44257c39680
I20260810 11:30:09.236493 8892 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 11:30:09.262689 8892 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 11:30:09.264281 8892 catalog_manager.cc:1504] Loading token signing keys...
I20260810 11:30:09.275177 8892 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f9409add55f141f2b144df70f51c2bbe: Generated new TSK 0
I20260810 11:30:09.276475 8892 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.18.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.18.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.18.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.193.keytab.
Entry for principal kudu/127.8.18.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.193.keytab.
I20260810 11:30:09.338828 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786361401922362-8267-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786361401922362-8267-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786361401922362-8267-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786361401922362-8267-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.18.193:0
--local_ip_for_outbound_sockets=127.8.18.193
--webserver_interface=127.8.18.193
--webserver_port=0
--tserver_master_addrs=127.8.18.254:35605
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.193.keytab
--principal=kudu/127.8.18.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.18.212:35165
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786361401922362-8267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786361401922362-8267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786361401922362-8267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:30:09.708796 8907 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:30:09.709127 8907 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:30:09.709312 8907 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:30:09.717955 8907 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:30:09.718092 8907 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:30:09.718307 8907 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.18.193
I20260810 11:30:09.729055 8907 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.18.212:35165
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786361401922362-8267-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786361401922362-8267-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.193.keytab
--principal=kudu/127.8.18.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.18.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786361401922362-8267-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.18.193
--webserver_port=0
--tserver_master_addrs=127.8.18.254:35605
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.18.193
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786361401922362-8267-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:30:09.730630 8907 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:30:09.732142 8907 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:30:09.747443 8913 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:30:09.747838 8912 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:30:09.747814 8915 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:30:09.748766 8907 server_base.cc:1034] running on GCE node
I20260810 11:30:09.749572 8907 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:30:09.750793 8907 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:30:09.752144 8907 hybrid_clock.cc:648] HybridClock initialized: now 1786361409752034 us; error 77 us; skew 500 ppm
I20260810 11:30:09.752599 8907 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:30:09 dist-test-slave-srgv krb5kdc[8801](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361409, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:30:09.756786 8907 init.cc:377] Logged in from keytab as kudu/127.8.18.193@KRBTEST.COM (short username kudu)
I20260810 11:30:09.758898 8907 webserver.cc:460] Webserver started at http://127.8.18.193:42317/ using document root <none> and password file <none>
I20260810 11:30:09.759826 8907 fs_manager.cc:359] Metadata directory not provided
I20260810 11:30:09.760020 8907 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:30:09.760427 8907 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:30:09.764257 8907 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786361401922362-8267-0/minicluster-data/ts-0/data/instance:
uuid: "14aaf2229fb64d958dcd3c5a567b736b"
format_stamp: "Formatted at 2026-08-10 11:30:09 on dist-test-slave-srgv"
server_key: "b7d72c849855de9d9232cdf696a20283"
server_key_iv: "14194f3515bf3fea2e8cca4c55d5ae68"
server_key_version: "encryptionkey@0"
I20260810 11:30:09.765321 8907 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786361401922362-8267-0/minicluster-data/ts-0/wal/instance:
uuid: "14aaf2229fb64d958dcd3c5a567b736b"
format_stamp: "Formatted at 2026-08-10 11:30:09 on dist-test-slave-srgv"
server_key: "b7d72c849855de9d9232cdf696a20283"
server_key_iv: "14194f3515bf3fea2e8cca4c55d5ae68"
server_key_version: "encryptionkey@0"
I20260810 11:30:09.772040 8907 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260810 11:30:09.777004 8922 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:30:09.778798 8907 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260810 11:30:09.779052 8907 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786361401922362-8267-0/minicluster-data/ts-0/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786361401922362-8267-0/minicluster-data/ts-0/wal
uuid: "14aaf2229fb64d958dcd3c5a567b736b"
format_stamp: "Formatted at 2026-08-10 11:30:09 on dist-test-slave-srgv"
server_key: "b7d72c849855de9d9232cdf696a20283"
server_key_iv: "14194f3515bf3fea2e8cca4c55d5ae68"
server_key_version: "encryptionkey@0"
I20260810 11:30:09.779327 8907 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786361401922362-8267-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786361401922362-8267-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786361401922362-8267-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:30:09.802718 8907 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:30:09.808501 8907 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:30:09.808918 8907 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:30:09.810230 8907 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:30:09.812072 8907 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:30:09.812224 8907 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:30:09.812436 8907 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:30:09.812530 8907 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:30:09.852994 8907 rpc_server.cc:307] RPC server started. Bound to: 127.8.18.193:34541
I20260810 11:30:09.853040 9034 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.18.193:34541 every 8 connection(s)
I20260810 11:30:09.854609 8907 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786361401922362-8267-0/minicluster-data/ts-0/data/info.pb
I20260810 11:30:09.862632 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 8907
I20260810 11:30:09.862903 8267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786361401922362-8267-0/minicluster-data/ts-0/wal/instance
I20260810 11:30:09.863667 8267 external_mini_cluster.cc:1428] Setting key 9dfd06aeb27ff4b7b818e7dcbc8828a9
Aug 10 11:30:09 dist-test-slave-srgv krb5kdc[8801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361409, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.193@KRBTEST.COM for kudu/127.8.18.254@KRBTEST.COM
I20260810 11:30:09.883639 9037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:09.858008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.193:44819 (local address 127.8.18.254:35605)
0810 11:30:09.858587 (+ 579us) server_negotiation.cc:206] Beginning negotiation
0810 11:30:09.858592 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:30:09.859672 (+ 1080us) server_negotiation.cc:407] Connection header received
0810 11:30:09.861846 (+ 2174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:30:09.861852 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:30:09.861962 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:30:09.862111 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:30:09.864364 (+ 2253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:09.865377 (+ 1013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:09.868391 (+ 3014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:09.868864 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:09.868997 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:09.874689 (+ 5692us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:30:09.874700 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:30:09.874709 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:30:09.874769 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:30:09.879087 (+ 4318us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:09.880214 (+ 1127us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:09.880224 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:09.880230 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:09.880420 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:09.881019 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:09.881028 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:09.881034 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:09.881525 (+ 491us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:30:09.881727 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0810 11:30:09.882761 (+ 1034us) server_negotiation.cc:299] Negotiation successful
0810 11:30:09.883091 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":473,"thread_start_us":329,"threads_started":1}
I20260810 11:30:09.884405 9038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:09.858587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:35605 (local address 127.8.18.193:44819)
0810 11:30:09.859446 (+ 859us) negotiation.cc:107] Waiting for socket to connect
0810 11:30:09.859504 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0810 11:30:09.861387 (+ 1883us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:30:09.862437 (+ 1050us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:30:09.862447 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:30:09.863187 (+ 740us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:30:09.864111 (+ 924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:30:09.864137 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:30:09.865650 (+ 1513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:30:09.865668 (+ 18us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:30:09.868100 (+ 2432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:30:09.868133 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:30:09.869201 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:30:09.869212 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:30:09.869685 (+ 473us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:09.870718 (+ 1033us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:30:09.870766 (+ 48us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:30:09.874327 (+ 3561us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:30:09.879363 (+ 5036us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:30:09.879372 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:30:09.879391 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:30:09.879993 (+ 602us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:30:09.880609 (+ 616us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:30:09.880619 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:30:09.880622 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:30:09.880865 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:30:09.881714 (+ 849us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:30:09.881721 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:30:09.882262 (+ 541us) client_negotiation.cc:769] Sending connection context
0810 11:30:09.882557 (+ 295us) client_negotiation.cc:240] Negotiation successful
0810 11:30:09.883033 (+ 476us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":567,"thread_start_us":328,"threads_started":1}
I20260810 11:30:09.886451 9035 heartbeater.cc:347] Connected to a master server at 127.8.18.254:35605
I20260810 11:30:09.886863 9035 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.8.18.194@KRBTEST.COM; defaulting to no policy
I20260810 11:30:09.887836 9035 heartbeater.cc:511] Master 127.8.18.254:35605 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.18.194@KRBTEST.COM" created.
I20260810 11:30:09.891034 8846 ts_manager.cc:194] Registered new tserver with Master: 14aaf2229fb64d958dcd3c5a567b736b (127.8.18.193:34541)
I20260810 11:30:09.893579 8846 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.18.193@KRBTEST.COM'} at 127.8.18.193:44819
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.18.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.194.keytab.
Entry for principal kudu/127.8.18.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.194.keytab.
I20260810 11:30:09.917929 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786361401922362-8267-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786361401922362-8267-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786361401922362-8267-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786361401922362-8267-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.18.194:0
--local_ip_for_outbound_sockets=127.8.18.194
--webserver_interface=127.8.18.194
--webserver_port=0
--tserver_master_addrs=127.8.18.254:35605
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.194.keytab
--principal=kudu/127.8.18.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.18.212:35165
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786361401922362-8267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786361401922362-8267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786361401922362-8267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:30:10.304662 9041 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:30:10.305105 9041 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:30:10.305342 9041 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:30:10.313759 9041 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:30:10.313947 9041 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:30:10.314150 9041 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.18.194
I20260810 11:30:10.325026 9041 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.18.212:35165
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786361401922362-8267-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786361401922362-8267-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.194.keytab
--principal=kudu/127.8.18.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.18.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786361401922362-8267-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.18.194
--webserver_port=0
--tserver_master_addrs=127.8.18.254:35605
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.18.194
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786361401922362-8267-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:30:10.326653 9041 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:30:10.328226 9041 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:30:10.344535 9047 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:30:10.344493 9046 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:30:10.344888 9049 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:30:10.345463 9041 server_base.cc:1034] running on GCE node
I20260810 11:30:10.346223 9041 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:30:10.347415 9041 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:30:10.348704 9041 hybrid_clock.cc:648] HybridClock initialized: now 1786361410348639 us; error 63 us; skew 500 ppm
I20260810 11:30:10.349170 9041 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:30:10 dist-test-slave-srgv krb5kdc[8801](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361410, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:30:10.353204 9041 init.cc:377] Logged in from keytab as kudu/127.8.18.194@KRBTEST.COM (short username kudu)
I20260810 11:30:10.355419 9041 webserver.cc:460] Webserver started at http://127.8.18.194:40203/ using document root <none> and password file <none>
I20260810 11:30:10.356300 9041 fs_manager.cc:359] Metadata directory not provided
I20260810 11:30:10.356416 9041 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:30:10.356716 9041 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:30:10.359993 9041 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786361401922362-8267-0/minicluster-data/ts-1/data/instance:
uuid: "c9006673c2d6438b8f84388a08260f30"
format_stamp: "Formatted at 2026-08-10 11:30:10 on dist-test-slave-srgv"
server_key: "996e2ceebbbe27540d226ec6e7896599"
server_key_iv: "94763f45260b08647a83c7b7eafd5b24"
server_key_version: "encryptionkey@0"
I20260810 11:30:10.360975 9041 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786361401922362-8267-0/minicluster-data/ts-1/wal/instance:
uuid: "c9006673c2d6438b8f84388a08260f30"
format_stamp: "Formatted at 2026-08-10 11:30:10 on dist-test-slave-srgv"
server_key: "996e2ceebbbe27540d226ec6e7896599"
server_key_iv: "94763f45260b08647a83c7b7eafd5b24"
server_key_version: "encryptionkey@0"
I20260810 11:30:10.367241 9041 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.003s
I20260810 11:30:10.371583 9056 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:30:10.373314 9041 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260810 11:30:10.373636 9041 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786361401922362-8267-0/minicluster-data/ts-1/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786361401922362-8267-0/minicluster-data/ts-1/wal
uuid: "c9006673c2d6438b8f84388a08260f30"
format_stamp: "Formatted at 2026-08-10 11:30:10 on dist-test-slave-srgv"
server_key: "996e2ceebbbe27540d226ec6e7896599"
server_key_iv: "94763f45260b08647a83c7b7eafd5b24"
server_key_version: "encryptionkey@0"
I20260810 11:30:10.373873 9041 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786361401922362-8267-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786361401922362-8267-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786361401922362-8267-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:30:10.392459 9041 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:30:10.398016 9041 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:30:10.398394 9041 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:30:10.399631 9041 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:30:10.401262 9041 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:30:10.401348 9041 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:30:10.401518 9041 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:30:10.401600 9041 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:30:10.439427 9041 rpc_server.cc:307] RPC server started. Bound to: 127.8.18.194:46139
I20260810 11:30:10.439478 9170 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.18.194:46139 every 8 connection(s)
I20260810 11:30:10.440945 9041 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786361401922362-8267-0/minicluster-data/ts-1/data/info.pb
I20260810 11:30:10.442551 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 9041
I20260810 11:30:10.442785 8267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786361401922362-8267-0/minicluster-data/ts-1/wal/instance
I20260810 11:30:10.443401 8267 external_mini_cluster.cc:1428] Setting key b34406c491940d7e270844eccda34fb3
Aug 10 11:30:10 dist-test-slave-srgv krb5kdc[8801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361410, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.194@KRBTEST.COM for kudu/127.8.18.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.8.18.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.18.195@KRBTEST.COM" created.
I20260810 11:30:10.465881 9173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:10.444912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.194:33889 (local address 127.8.18.254:35605)
0810 11:30:10.445483 (+ 571us) server_negotiation.cc:206] Beginning negotiation
0810 11:30:10.445491 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:30:10.446627 (+ 1136us) server_negotiation.cc:407] Connection header received
0810 11:30:10.448859 (+ 2232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:30:10.448865 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:30:10.448975 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:30:10.449105 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:30:10.451445 (+ 2340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:10.452191 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:10.454049 (+ 1858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:10.454517 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:10.454669 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:10.458923 (+ 4254us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:30:10.458929 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:30:10.458932 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:30:10.458971 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:30:10.461995 (+ 3024us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:10.462990 (+ 995us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:10.462995 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:10.462997 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:10.463171 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:10.463757 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:10.463763 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:10.463771 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:10.464085 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:30:10.464265 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0810 11:30:10.465143 (+ 878us) server_negotiation.cc:299] Negotiation successful
0810 11:30:10.465471 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":409,"thread_start_us":267,"threads_started":1}
I20260810 11:30:10.467283 9174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:10.445439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:35605 (local address 127.8.18.194:33889)
0810 11:30:10.446437 (+ 998us) negotiation.cc:107] Waiting for socket to connect
0810 11:30:10.446492 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0810 11:30:10.448486 (+ 1994us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:30:10.449488 (+ 1002us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:30:10.449506 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:30:10.450451 (+ 945us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:30:10.451209 (+ 758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:30:10.451233 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:30:10.452417 (+ 1184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:30:10.452423 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:30:10.453868 (+ 1445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:30:10.453884 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:30:10.454802 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:30:10.454810 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:30:10.455069 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:10.455827 (+ 758us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:30:10.455853 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:30:10.458587 (+ 2734us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:30:10.462201 (+ 3614us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:30:10.462212 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:30:10.462235 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:30:10.462827 (+ 592us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:30:10.463311 (+ 484us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:30:10.463317 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:30:10.463321 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:30:10.463559 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:30:10.464229 (+ 670us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:30:10.464238 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:30:10.464657 (+ 419us) client_negotiation.cc:769] Sending connection context
0810 11:30:10.465037 (+ 380us) client_negotiation.cc:240] Negotiation successful
0810 11:30:10.465805 (+ 768us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":656,"thread_start_us":412,"threads_started":1}
I20260810 11:30:10.468983 9171 heartbeater.cc:347] Connected to a master server at 127.8.18.254:35605
I20260810 11:30:10.469362 9171 heartbeater.cc:464] Registering TS with master...
I20260810 11:30:10.470196 9171 heartbeater.cc:511] Master 127.8.18.254:35605 requested a full tablet report, sending...
I20260810 11:30:10.471854 8846 ts_manager.cc:194] Registered new tserver with Master: c9006673c2d6438b8f84388a08260f30 (127.8.18.194:46139)
I20260810 11:30:10.472757 8846 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.18.194@KRBTEST.COM'} at 127.8.18.194:33889
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.18.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.195.keytab.
Entry for principal kudu/127.8.18.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.195.keytab.
I20260810 11:30:10.493696 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786361401922362-8267-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786361401922362-8267-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786361401922362-8267-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786361401922362-8267-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.18.195:0
--local_ip_for_outbound_sockets=127.8.18.195
--webserver_interface=127.8.18.195
--webserver_port=0
--tserver_master_addrs=127.8.18.254:35605
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.195.keytab
--principal=kudu/127.8.18.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.18.212:35165
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786361401922362-8267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786361401922362-8267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786361401922362-8267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:30:10.852515 9177 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:30:10.852833 9177 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:30:10.853000 9177 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:30:10.860971 9177 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:30:10.861127 9177 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:30:10.861249 9177 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.18.195
I20260810 11:30:10.872403 9177 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.18.212:35165
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786361401922362-8267-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786361401922362-8267-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.195.keytab
--principal=kudu/127.8.18.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.18.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786361401922362-8267-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.18.195
--webserver_port=0
--tserver_master_addrs=127.8.18.254:35605
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.18.195
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786361401922362-8267-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:30:10.874042 9177 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:30:10.875545 9177 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:30:10.891069 9183 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:30:10.891228 9182 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:30:10.891249 9185 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:30:10.892509 9177 server_base.cc:1034] running on GCE node
I20260810 11:30:10.893513 9177 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:30:10.894723 9177 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:30:10.895970 9177 hybrid_clock.cc:648] HybridClock initialized: now 1786361410895920 us; error 43 us; skew 500 ppm
I20260810 11:30:10.896422 9177 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 11:30:10.898140 9035 heartbeater.cc:503] Master 127.8.18.254:35605 was elected leader, sending a full tablet report...
Aug 10 11:30:10 dist-test-slave-srgv krb5kdc[8801](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361410, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:30:10.900768 9177 init.cc:377] Logged in from keytab as kudu/127.8.18.195@KRBTEST.COM (short username kudu)
I20260810 11:30:10.902814 9177 webserver.cc:460] Webserver started at http://127.8.18.195:41333/ using document root <none> and password file <none>
I20260810 11:30:10.903695 9177 fs_manager.cc:359] Metadata directory not provided
I20260810 11:30:10.903869 9177 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:30:10.904304 9177 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:30:10.908015 9177 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786361401922362-8267-0/minicluster-data/ts-2/data/instance:
uuid: "3516618266c8490dbd101f35d519fda1"
format_stamp: "Formatted at 2026-08-10 11:30:10 on dist-test-slave-srgv"
server_key: "43e8ba3fb59bde286c7891775bf1b08e"
server_key_iv: "666204d62d95f7bab8f57ebdd5dc76fc"
server_key_version: "encryptionkey@0"
I20260810 11:30:10.908981 9177 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786361401922362-8267-0/minicluster-data/ts-2/wal/instance:
uuid: "3516618266c8490dbd101f35d519fda1"
format_stamp: "Formatted at 2026-08-10 11:30:10 on dist-test-slave-srgv"
server_key: "43e8ba3fb59bde286c7891775bf1b08e"
server_key_iv: "666204d62d95f7bab8f57ebdd5dc76fc"
server_key_version: "encryptionkey@0"
I20260810 11:30:10.915551 9177 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260810 11:30:10.920361 9193 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:30:10.922380 9177 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260810 11:30:10.922572 9177 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786361401922362-8267-0/minicluster-data/ts-2/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786361401922362-8267-0/minicluster-data/ts-2/wal
uuid: "3516618266c8490dbd101f35d519fda1"
format_stamp: "Formatted at 2026-08-10 11:30:10 on dist-test-slave-srgv"
server_key: "43e8ba3fb59bde286c7891775bf1b08e"
server_key_iv: "666204d62d95f7bab8f57ebdd5dc76fc"
server_key_version: "encryptionkey@0"
I20260810 11:30:10.922784 9177 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786361401922362-8267-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786361401922362-8267-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786361401922362-8267-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:30:10.945708 9177 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:30:10.950917 9177 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:30:10.951213 9177 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:30:10.952360 9177 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:30:10.953922 9177 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:30:10.954007 9177 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:30:10.954109 9177 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:30:10.954155 9177 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:30:11.000157 9177 rpc_server.cc:307] RPC server started. Bound to: 127.8.18.195:40631
I20260810 11:30:11.000288 9306 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.18.195:40631 every 8 connection(s)
I20260810 11:30:11.001926 9177 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786361401922362-8267-0/minicluster-data/ts-2/data/info.pb
I20260810 11:30:11.006733 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 9177
I20260810 11:30:11.007000 8267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786361401922362-8267-0/minicluster-data/ts-2/wal/instance
I20260810 11:30:11.007624 8267 external_mini_cluster.cc:1428] Setting key 69c290159fb1f4024652bb5d71db9aa4
Aug 10 11:30:11 dist-test-slave-srgv krb5kdc[8801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361410, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.195@KRBTEST.COM for kudu/127.8.18.254@KRBTEST.COM
I20260810 11:30:11.030154 9309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:11.005270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.195:52699 (local address 127.8.18.254:35605)
0810 11:30:11.006032 (+ 762us) server_negotiation.cc:206] Beginning negotiation
0810 11:30:11.006037 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:30:11.007313 (+ 1276us) server_negotiation.cc:407] Connection header received
0810 11:30:11.010005 (+ 2692us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:30:11.010009 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:30:11.010132 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:30:11.010258 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:30:11.012459 (+ 2201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:11.013766 (+ 1307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:11.015656 (+ 1890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:11.016153 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:11.016269 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:11.021466 (+ 5197us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:30:11.021474 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:30:11.021478 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:30:11.021561 (+ 83us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:30:11.025856 (+ 4295us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:11.026918 (+ 1062us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:11.026923 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:11.026925 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:11.027126 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:11.027717 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:11.027722 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:11.027725 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:11.028228 (+ 503us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:30:11.028432 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0810 11:30:11.029302 (+ 870us) server_negotiation.cc:299] Negotiation successful
0810 11:30:11.029608 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":653,"thread_start_us":472,"threads_started":1}
I20260810 11:30:11.031185 9310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:11.005890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:35605 (local address 127.8.18.195:52699)
0810 11:30:11.007095 (+ 1205us) negotiation.cc:107] Waiting for socket to connect
0810 11:30:11.007158 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0810 11:30:11.009663 (+ 2505us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:30:11.010511 (+ 848us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:30:11.010521 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:30:11.011417 (+ 896us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:30:11.012229 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:30:11.012265 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:30:11.013958 (+ 1693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:30:11.013963 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:30:11.015471 (+ 1508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:30:11.015486 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:30:11.016521 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:30:11.016529 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:30:11.016862 (+ 333us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:11.017833 (+ 971us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:30:11.017881 (+ 48us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:30:11.021052 (+ 3171us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:30:11.026048 (+ 4996us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:30:11.026057 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:30:11.026074 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:30:11.026723 (+ 649us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:30:11.027289 (+ 566us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:30:11.027293 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:30:11.027295 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:30:11.027547 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:30:11.028395 (+ 848us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:30:11.028405 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:30:11.028871 (+ 466us) client_negotiation.cc:769] Sending connection context
0810 11:30:11.029186 (+ 315us) client_negotiation.cc:240] Negotiation successful
0810 11:30:11.029837 (+ 651us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":689,"thread_start_us":460,"threads_started":1}
I20260810 11:30:11.033150 9307 heartbeater.cc:347] Connected to a master server at 127.8.18.254:35605
I20260810 11:30:11.033623 9307 heartbeater.cc:464] Registering TS with master...
I20260810 11:30:11.034462 9307 heartbeater.cc:511] Master 127.8.18.254:35605 requested a full tablet report, sending...
I20260810 11:30:11.036414 8845 ts_manager.cc:194] Registered new tserver with Master: 3516618266c8490dbd101f35d519fda1 (127.8.18.195:40631)
I20260810 11:30:11.037783 8267 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 11:30:11.037808 8845 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.18.195@KRBTEST.COM'} at 127.8.18.195:52699
I20260810 11:30:11.062981 9309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:11.045882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48670 (local address 127.8.18.254:35605)
0810 11:30:11.046139 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0810 11:30:11.046145 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:30:11.047123 (+ 978us) server_negotiation.cc:407] Connection header received
0810 11:30:11.047286 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:30:11.047290 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:30:11.047367 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:30:11.047466 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:30:11.049571 (+ 2105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:11.050487 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:11.052137 (+ 1650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:11.052660 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:11.052790 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:11.056007 (+ 3217us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:30:11.056013 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:30:11.056016 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:30:11.056111 (+ 95us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:30:11.059547 (+ 3436us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:11.060509 (+ 962us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:11.060514 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:11.060516 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:11.060703 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:11.061243 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:11.061247 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:11.061249 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:11.061810 (+ 561us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:30:11.062036 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0810 11:30:11.062460 (+ 424us) server_negotiation.cc:299] Negotiation successful
0810 11:30:11.062683 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260810 11:30:11.073778 8845 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48670:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 11:30:11.085098 8845 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 11:30:11.121928 9320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:11.107368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.195:40631 (local address 127.0.0.1:40380)
0810 11:30:11.108663 (+ 1295us) negotiation.cc:107] Waiting for socket to connect
0810 11:30:11.108767 (+ 104us) client_negotiation.cc:174] Beginning negotiation
0810 11:30:11.109193 (+ 426us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:30:11.111931 (+ 2738us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:30:11.111943 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:30:11.111989 (+ 46us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:30:11.112550 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:30:11.112566 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:30:11.114547 (+ 1981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:30:11.114553 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:30:11.117745 (+ 3192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:30:11.117760 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:30:11.120128 (+ 2368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:30:11.120137 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:30:11.120581 (+ 444us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:11.120660 (+ 79us) client_negotiation.cc:769] Sending connection context
0810 11:30:11.120816 (+ 156us) client_negotiation.cc:240] Negotiation successful
0810 11:30:11.121097 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":810,"mutex_wait_us":138,"spinlock_wait_cycles":896,"thread_start_us":776,"threads_started":1}
I20260810 11:30:11.122620 9321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:11.107439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40380 (local address 127.8.18.195:40631)
0810 11:30:11.110170 (+ 2731us) server_negotiation.cc:206] Beginning negotiation
0810 11:30:11.110177 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:30:11.110206 (+ 29us) server_negotiation.cc:407] Connection header received
0810 11:30:11.110329 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:30:11.110341 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:30:11.110653 (+ 312us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:30:11.111887 (+ 1234us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:30:11.112851 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:11.114371 (+ 1520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:11.117945 (+ 3574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:11.119818 (+ 1873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:11.119961 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:11.121800 (+ 1839us) server_negotiation.cc:1035] Waiting for connection context
0810 11:30:11.121904 (+ 104us) server_negotiation.cc:299] Negotiation successful
0810 11:30:11.122110 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2504,"thread_start_us":2354,"threads_started":1}
I20260810 11:30:11.126181 9325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:11.110025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45068 (local address 127.8.18.193:34541)
0810 11:30:11.110802 (+ 777us) server_negotiation.cc:206] Beginning negotiation
0810 11:30:11.110811 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 11:30:11.114127 (+ 3316us) server_negotiation.cc:407] Connection header received
0810 11:30:11.114371 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:30:11.114381 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:30:11.114727 (+ 346us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:30:11.114971 (+ 244us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:30:11.116217 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:11.117965 (+ 1748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:11.120898 (+ 2933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:11.122558 (+ 1660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:11.122730 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:11.122981 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0810 11:30:11.125143 (+ 2162us) server_negotiation.cc:299] Negotiation successful
0810 11:30:11.125403 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":602,"thread_start_us":375,"threads_started":1}
I20260810 11:30:11.126238 9324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:11.109125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.193:34541 (local address 127.0.0.1:45068)
0810 11:30:11.113947 (+ 4822us) negotiation.cc:107] Waiting for socket to connect
0810 11:30:11.113973 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0810 11:30:11.114151 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:30:11.114976 (+ 825us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:30:11.114983 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:30:11.115009 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:30:11.115883 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:30:11.115903 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:30:11.118143 (+ 2240us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:30:11.118148 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:30:11.120665 (+ 2517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:30:11.120683 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:30:11.124475 (+ 3792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:30:11.124485 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:30:11.124727 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:11.124762 (+ 35us) client_negotiation.cc:769] Sending connection context
0810 11:30:11.124986 (+ 224us) client_negotiation.cc:240] Negotiation successful
0810 11:30:11.125160 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4637,"mutex_wait_us":27,"thread_start_us":4492,"threads_started":1}
I20260810 11:30:11.127822 9322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:11.107533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.194:46139 (local address 127.0.0.1:46644)
0810 11:30:11.111542 (+ 4009us) negotiation.cc:107] Waiting for socket to connect
0810 11:30:11.111566 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0810 11:30:11.111669 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:30:11.115606 (+ 3937us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:30:11.115613 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:30:11.115642 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:30:11.116330 (+ 688us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:30:11.116341 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:30:11.118115 (+ 1774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:30:11.118122 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:30:11.120473 (+ 2351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:30:11.120494 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:30:11.124380 (+ 3886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:30:11.124405 (+ 25us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:30:11.124825 (+ 420us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:11.124860 (+ 35us) client_negotiation.cc:769] Sending connection context
0810 11:30:11.124968 (+ 108us) client_negotiation.cc:240] Negotiation successful
0810 11:30:11.125142 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3889,"mutex_wait_us":20,"thread_start_us":3746,"threads_started":1}
I20260810 11:30:11.129055 9323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:11.107974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46644 (local address 127.8.18.194:46139)
0810 11:30:11.114701 (+ 6727us) server_negotiation.cc:206] Beginning negotiation
0810 11:30:11.114709 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:30:11.114745 (+ 36us) server_negotiation.cc:407] Connection header received
0810 11:30:11.114821 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:30:11.114827 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:30:11.115088 (+ 261us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:30:11.115347 (+ 259us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:30:11.116547 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:11.117942 (+ 1395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:11.120742 (+ 2800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:11.123664 (+ 2922us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:11.123892 (+ 228us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:11.124117 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0810 11:30:11.128268 (+ 4151us) server_negotiation.cc:299] Negotiation successful
0810 11:30:11.128572 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6572,"thread_start_us":6464,"threads_started":1}
I20260810 11:30:11.128750 9241 tablet_service.cc:1467] Processing CreateTablet for tablet 3251aee2986f46a4a7c1d97ab8993d85 (DEFAULT_TABLE table=test-table [id=04049190b8a6444f8e8326e2d64ce0c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:30:11.130918 9241 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3251aee2986f46a4a7c1d97ab8993d85. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:30:11.131311 8970 tablet_service.cc:1467] Processing CreateTablet for tablet 3251aee2986f46a4a7c1d97ab8993d85 (DEFAULT_TABLE table=test-table [id=04049190b8a6444f8e8326e2d64ce0c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:30:11.133297 8970 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3251aee2986f46a4a7c1d97ab8993d85. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:30:11.136132 9104 tablet_service.cc:1467] Processing CreateTablet for tablet 3251aee2986f46a4a7c1d97ab8993d85 (DEFAULT_TABLE table=test-table [id=04049190b8a6444f8e8326e2d64ce0c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:30:11.137693 9104 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3251aee2986f46a4a7c1d97ab8993d85. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:30:11.145346 9326 tablet_bootstrap.cc:492] T 3251aee2986f46a4a7c1d97ab8993d85 P 3516618266c8490dbd101f35d519fda1: Bootstrap starting.
I20260810 11:30:11.149330 9326 tablet_bootstrap.cc:654] T 3251aee2986f46a4a7c1d97ab8993d85 P 3516618266c8490dbd101f35d519fda1: Neither blocks nor log segments found. Creating new log.
I20260810 11:30:11.150945 9326 log.cc:824] T 3251aee2986f46a4a7c1d97ab8993d85 P 3516618266c8490dbd101f35d519fda1: Log is configured to *not* fsync() on all Append() calls
I20260810 11:30:11.154543 9326 tablet_bootstrap.cc:492] T 3251aee2986f46a4a7c1d97ab8993d85 P 3516618266c8490dbd101f35d519fda1: No bootstrap required, opened a new log
I20260810 11:30:11.154923 9326 ts_tablet_manager.cc:1397] T 3251aee2986f46a4a7c1d97ab8993d85 P 3516618266c8490dbd101f35d519fda1: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.004s
I20260810 11:30:11.156404 9328 tablet_bootstrap.cc:492] T 3251aee2986f46a4a7c1d97ab8993d85 P 14aaf2229fb64d958dcd3c5a567b736b: Bootstrap starting.
I20260810 11:30:11.158102 9329 tablet_bootstrap.cc:492] T 3251aee2986f46a4a7c1d97ab8993d85 P c9006673c2d6438b8f84388a08260f30: Bootstrap starting.
I20260810 11:30:11.164081 9328 tablet_bootstrap.cc:654] T 3251aee2986f46a4a7c1d97ab8993d85 P 14aaf2229fb64d958dcd3c5a567b736b: Neither blocks nor log segments found. Creating new log.
I20260810 11:30:11.164321 9329 tablet_bootstrap.cc:654] T 3251aee2986f46a4a7c1d97ab8993d85 P c9006673c2d6438b8f84388a08260f30: Neither blocks nor log segments found. Creating new log.
I20260810 11:30:11.165828 9328 log.cc:824] T 3251aee2986f46a4a7c1d97ab8993d85 P 14aaf2229fb64d958dcd3c5a567b736b: Log is configured to *not* fsync() on all Append() calls
I20260810 11:30:11.165956 9329 log.cc:824] T 3251aee2986f46a4a7c1d97ab8993d85 P c9006673c2d6438b8f84388a08260f30: Log is configured to *not* fsync() on all Append() calls
I20260810 11:30:11.170732 9326 raft_consensus.cc:348] T 3251aee2986f46a4a7c1d97ab8993d85 P 3516618266c8490dbd101f35d519fda1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3516618266c8490dbd101f35d519fda1" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 40631 } } peers { permanent_uuid: "c9006673c2d6438b8f84388a08260f30" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 46139 } } peers { permanent_uuid: "14aaf2229fb64d958dcd3c5a567b736b" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 34541 } }
I20260810 11:30:11.171531 9326 raft_consensus.cc:374] T 3251aee2986f46a4a7c1d97ab8993d85 P 3516618266c8490dbd101f35d519fda1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:30:11.171708 9326 raft_consensus.cc:729] T 3251aee2986f46a4a7c1d97ab8993d85 P 3516618266c8490dbd101f35d519fda1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3516618266c8490dbd101f35d519fda1, State: Initialized, Role: FOLLOWER
I20260810 11:30:11.172837 9326 consensus_queue.cc:260] T 3251aee2986f46a4a7c1d97ab8993d85 P 3516618266c8490dbd101f35d519fda1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3516618266c8490dbd101f35d519fda1" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 40631 } } peers { permanent_uuid: "c9006673c2d6438b8f84388a08260f30" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 46139 } } peers { permanent_uuid: "14aaf2229fb64d958dcd3c5a567b736b" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 34541 } }
I20260810 11:30:11.178287 9328 tablet_bootstrap.cc:492] T 3251aee2986f46a4a7c1d97ab8993d85 P 14aaf2229fb64d958dcd3c5a567b736b: No bootstrap required, opened a new log
I20260810 11:30:11.178817 9329 tablet_bootstrap.cc:492] T 3251aee2986f46a4a7c1d97ab8993d85 P c9006673c2d6438b8f84388a08260f30: No bootstrap required, opened a new log
I20260810 11:30:11.179016 9307 heartbeater.cc:503] Master 127.8.18.254:35605 was elected leader, sending a full tablet report...
I20260810 11:30:11.179208 9329 ts_tablet_manager.cc:1397] T 3251aee2986f46a4a7c1d97ab8993d85 P c9006673c2d6438b8f84388a08260f30: Time spent bootstrapping tablet: real 0.021s user 0.004s sys 0.015s
I20260810 11:30:11.179767 9326 ts_tablet_manager.cc:1428] T 3251aee2986f46a4a7c1d97ab8993d85 P 3516618266c8490dbd101f35d519fda1: Time spent starting tablet: real 0.025s user 0.020s sys 0.003s
I20260810 11:30:11.178689 9328 ts_tablet_manager.cc:1397] T 3251aee2986f46a4a7c1d97ab8993d85 P 14aaf2229fb64d958dcd3c5a567b736b: Time spent bootstrapping tablet: real 0.023s user 0.004s sys 0.016s
I20260810 11:30:11.193277 9328 raft_consensus.cc:348] T 3251aee2986f46a4a7c1d97ab8993d85 P 14aaf2229fb64d958dcd3c5a567b736b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3516618266c8490dbd101f35d519fda1" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 40631 } } peers { permanent_uuid: "c9006673c2d6438b8f84388a08260f30" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 46139 } } peers { permanent_uuid: "14aaf2229fb64d958dcd3c5a567b736b" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 34541 } }
I20260810 11:30:11.193277 9329 raft_consensus.cc:348] T 3251aee2986f46a4a7c1d97ab8993d85 P c9006673c2d6438b8f84388a08260f30 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3516618266c8490dbd101f35d519fda1" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 40631 } } peers { permanent_uuid: "c9006673c2d6438b8f84388a08260f30" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 46139 } } peers { permanent_uuid: "14aaf2229fb64d958dcd3c5a567b736b" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 34541 } }
I20260810 11:30:11.193835 9328 raft_consensus.cc:374] T 3251aee2986f46a4a7c1d97ab8993d85 P 14aaf2229fb64d958dcd3c5a567b736b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:30:11.193838 9329 raft_consensus.cc:374] T 3251aee2986f46a4a7c1d97ab8993d85 P c9006673c2d6438b8f84388a08260f30 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:30:11.193949 9328 raft_consensus.cc:729] T 3251aee2986f46a4a7c1d97ab8993d85 P 14aaf2229fb64d958dcd3c5a567b736b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14aaf2229fb64d958dcd3c5a567b736b, State: Initialized, Role: FOLLOWER
I20260810 11:30:11.193948 9329 raft_consensus.cc:729] T 3251aee2986f46a4a7c1d97ab8993d85 P c9006673c2d6438b8f84388a08260f30 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c9006673c2d6438b8f84388a08260f30, State: Initialized, Role: FOLLOWER
I20260810 11:30:11.194667 9329 consensus_queue.cc:260] T 3251aee2986f46a4a7c1d97ab8993d85 P c9006673c2d6438b8f84388a08260f30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3516618266c8490dbd101f35d519fda1" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 40631 } } peers { permanent_uuid: "c9006673c2d6438b8f84388a08260f30" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 46139 } } peers { permanent_uuid: "14aaf2229fb64d958dcd3c5a567b736b" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 34541 } }
I20260810 11:30:11.194752 9328 consensus_queue.cc:260] T 3251aee2986f46a4a7c1d97ab8993d85 P 14aaf2229fb64d958dcd3c5a567b736b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3516618266c8490dbd101f35d519fda1" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 40631 } } peers { permanent_uuid: "c9006673c2d6438b8f84388a08260f30" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 46139 } } peers { permanent_uuid: "14aaf2229fb64d958dcd3c5a567b736b" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 34541 } }
I20260810 11:30:11.195994 9171 heartbeater.cc:503] Master 127.8.18.254:35605 was elected leader, sending a full tablet report...
I20260810 11:30:11.196799 9328 ts_tablet_manager.cc:1428] T 3251aee2986f46a4a7c1d97ab8993d85 P 14aaf2229fb64d958dcd3c5a567b736b: Time spent starting tablet: real 0.016s user 0.015s sys 0.000s
I20260810 11:30:11.196817 9329 ts_tablet_manager.cc:1428] T 3251aee2986f46a4a7c1d97ab8993d85 P c9006673c2d6438b8f84388a08260f30: Time spent starting tablet: real 0.017s user 0.016s sys 0.000s
W20260810 11:30:11.254438 9308 tablet.cc:2369] T 3251aee2986f46a4a7c1d97ab8993d85 P 3516618266c8490dbd101f35d519fda1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 11:30:11.278808 9334 raft_consensus.cc:482] T 3251aee2986f46a4a7c1d97ab8993d85 P 14aaf2229fb64d958dcd3c5a567b736b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 11:30:11.279213 9334 raft_consensus.cc:504] T 3251aee2986f46a4a7c1d97ab8993d85 P 14aaf2229fb64d958dcd3c5a567b736b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3516618266c8490dbd101f35d519fda1" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 40631 } } peers { permanent_uuid: "c9006673c2d6438b8f84388a08260f30" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 46139 } } peers { permanent_uuid: "14aaf2229fb64d958dcd3c5a567b736b" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 34541 } }
I20260810 11:30:11.281827 9334 leader_election.cc:288] T 3251aee2986f46a4a7c1d97ab8993d85 P 14aaf2229fb64d958dcd3c5a567b736b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3516618266c8490dbd101f35d519fda1 (127.8.18.195:40631), c9006673c2d6438b8f84388a08260f30 (127.8.18.194:46139)
I20260810 11:30:11.291255 9321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:11.282152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.193:38513 (local address 127.8.18.195:40631)
0810 11:30:11.282391 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0810 11:30:11.282397 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:30:11.283574 (+ 1177us) server_negotiation.cc:407] Connection header received
0810 11:30:11.283818 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:30:11.283823 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:30:11.283909 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:30:11.284048 (+ 139us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:30:11.285208 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:11.286310 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:11.288841 (+ 2531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:11.289938 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:11.290177 (+ 239us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:11.290270 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0810 11:30:11.290729 (+ 459us) server_negotiation.cc:299] Negotiation successful
0810 11:30:11.290911 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260810 11:30:11.291285 9323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:11.282493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.193:34823 (local address 127.8.18.194:46139)
0810 11:30:11.282786 (+ 293us) server_negotiation.cc:206] Beginning negotiation
0810 11:30:11.282796 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 11:30:11.283754 (+ 958us) server_negotiation.cc:407] Connection header received
0810 11:30:11.283833 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:30:11.283837 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:30:11.283910 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:30:11.284048 (+ 138us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:30:11.285522 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:11.286308 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:11.288803 (+ 2495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:11.289871 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:11.290097 (+ 226us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:11.290194 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0810 11:30:11.290711 (+ 517us) server_negotiation.cc:299] Negotiation successful
0810 11:30:11.290884 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260810 11:30:11.291576 9336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:11.282353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.194:46139 (local address 127.8.18.193:34823)
0810 11:30:11.283528 (+ 1175us) negotiation.cc:107] Waiting for socket to connect
0810 11:30:11.283544 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 11:30:11.283659 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:30:11.284109 (+ 450us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:30:11.284113 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:30:11.284220 (+ 107us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:30:11.285282 (+ 1062us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:30:11.285294 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:30:11.286512 (+ 1218us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:30:11.286516 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:30:11.288503 (+ 1987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:30:11.288526 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:30:11.290213 (+ 1687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:30:11.290224 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:30:11.290463 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:11.290528 (+ 65us) client_negotiation.cc:769] Sending connection context
0810 11:30:11.290668 (+ 140us) client_negotiation.cc:240] Negotiation successful
0810 11:30:11.290903 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1027,"mutex_wait_us":54,"spinlock_wait_cycles":28032,"thread_start_us":513,"threads_started":1}
I20260810 11:30:11.291836 9335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:11.281989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.195:40631 (local address 127.8.18.193:38513)
0810 11:30:11.283393 (+ 1404us) negotiation.cc:107] Waiting for socket to connect
0810 11:30:11.283420 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0810 11:30:11.283648 (+ 228us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:30:11.284084 (+ 436us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:30:11.284091 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:30:11.284139 (+ 48us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:30:11.284823 (+ 684us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:30:11.284852 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:30:11.286487 (+ 1635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:30:11.286496 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:30:11.288576 (+ 2080us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:30:11.288594 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:30:11.290273 (+ 1679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:30:11.290280 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:30:11.290508 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:11.290535 (+ 27us) client_negotiation.cc:769] Sending connection context
0810 11:30:11.290668 (+ 133us) client_negotiation.cc:240] Negotiation successful
0810 11:30:11.290928 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1212,"mutex_wait_us":70,"thread_start_us":875,"threads_started":1}
I20260810 11:30:11.293004 9261 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3251aee2986f46a4a7c1d97ab8993d85" candidate_uuid: "14aaf2229fb64d958dcd3c5a567b736b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3516618266c8490dbd101f35d519fda1" is_pre_election: true
I20260810 11:30:11.293005 9125 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3251aee2986f46a4a7c1d97ab8993d85" candidate_uuid: "14aaf2229fb64d958dcd3c5a567b736b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c9006673c2d6438b8f84388a08260f30" is_pre_election: true
I20260810 11:30:11.293983 9125 raft_consensus.cc:2454] T 3251aee2986f46a4a7c1d97ab8993d85 P c9006673c2d6438b8f84388a08260f30 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 14aaf2229fb64d958dcd3c5a567b736b in term 0.
I20260810 11:30:11.294917 8924 leader_election.cc:302] T 3251aee2986f46a4a7c1d97ab8993d85 P 14aaf2229fb64d958dcd3c5a567b736b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 14aaf2229fb64d958dcd3c5a567b736b, c9006673c2d6438b8f84388a08260f30; no voters:
I20260810 11:30:11.295367 9334 raft_consensus.cc:2781] T 3251aee2986f46a4a7c1d97ab8993d85 P 14aaf2229fb64d958dcd3c5a567b736b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 11:30:11.295567 9334 raft_consensus.cc:482] T 3251aee2986f46a4a7c1d97ab8993d85 P 14aaf2229fb64d958dcd3c5a567b736b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 11:30:11.295745 9334 raft_consensus.cc:3037] T 3251aee2986f46a4a7c1d97ab8993d85 P 14aaf2229fb64d958dcd3c5a567b736b [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:30:11.296131 9261 raft_consensus.cc:2454] T 3251aee2986f46a4a7c1d97ab8993d85 P 3516618266c8490dbd101f35d519fda1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 14aaf2229fb64d958dcd3c5a567b736b in term 0.
I20260810 11:30:11.298873 9334 raft_consensus.cc:504] T 3251aee2986f46a4a7c1d97ab8993d85 P 14aaf2229fb64d958dcd3c5a567b736b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3516618266c8490dbd101f35d519fda1" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 40631 } } peers { permanent_uuid: "c9006673c2d6438b8f84388a08260f30" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 46139 } } peers { permanent_uuid: "14aaf2229fb64d958dcd3c5a567b736b" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 34541 } }
I20260810 11:30:11.299747 9334 leader_election.cc:288] T 3251aee2986f46a4a7c1d97ab8993d85 P 14aaf2229fb64d958dcd3c5a567b736b [CANDIDATE]: Term 1 election: Requested vote from peers 3516618266c8490dbd101f35d519fda1 (127.8.18.195:40631), c9006673c2d6438b8f84388a08260f30 (127.8.18.194:46139)
I20260810 11:30:11.300354 9261 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3251aee2986f46a4a7c1d97ab8993d85" candidate_uuid: "14aaf2229fb64d958dcd3c5a567b736b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3516618266c8490dbd101f35d519fda1"
I20260810 11:30:11.300530 9125 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3251aee2986f46a4a7c1d97ab8993d85" candidate_uuid: "14aaf2229fb64d958dcd3c5a567b736b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c9006673c2d6438b8f84388a08260f30"
I20260810 11:30:11.300637 9261 raft_consensus.cc:3037] T 3251aee2986f46a4a7c1d97ab8993d85 P 3516618266c8490dbd101f35d519fda1 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:30:11.300818 9125 raft_consensus.cc:3037] T 3251aee2986f46a4a7c1d97ab8993d85 P c9006673c2d6438b8f84388a08260f30 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:30:11.304750 9261 raft_consensus.cc:2454] T 3251aee2986f46a4a7c1d97ab8993d85 P 3516618266c8490dbd101f35d519fda1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 14aaf2229fb64d958dcd3c5a567b736b in term 1.
I20260810 11:30:11.304774 9125 raft_consensus.cc:2454] T 3251aee2986f46a4a7c1d97ab8993d85 P c9006673c2d6438b8f84388a08260f30 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 14aaf2229fb64d958dcd3c5a567b736b in term 1.
I20260810 11:30:11.305379 8924 leader_election.cc:302] T 3251aee2986f46a4a7c1d97ab8993d85 P 14aaf2229fb64d958dcd3c5a567b736b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 14aaf2229fb64d958dcd3c5a567b736b, c9006673c2d6438b8f84388a08260f30; no voters:
I20260810 11:30:11.305886 9334 raft_consensus.cc:2781] T 3251aee2986f46a4a7c1d97ab8993d85 P 14aaf2229fb64d958dcd3c5a567b736b [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:30:11.306545 9334 raft_consensus.cc:686] T 3251aee2986f46a4a7c1d97ab8993d85 P 14aaf2229fb64d958dcd3c5a567b736b [term 1 LEADER]: Becoming Leader. State: Replica: 14aaf2229fb64d958dcd3c5a567b736b, State: Running, Role: LEADER
I20260810 11:30:11.307196 9334 consensus_queue.cc:237] T 3251aee2986f46a4a7c1d97ab8993d85 P 14aaf2229fb64d958dcd3c5a567b736b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3516618266c8490dbd101f35d519fda1" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 40631 } } peers { permanent_uuid: "c9006673c2d6438b8f84388a08260f30" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 46139 } } peers { permanent_uuid: "14aaf2229fb64d958dcd3c5a567b736b" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 34541 } }
I20260810 11:30:11.313875 8845 catalog_manager.cc:5495] T 3251aee2986f46a4a7c1d97ab8993d85 P 14aaf2229fb64d958dcd3c5a567b736b reported cstate change: term changed from 0 to 1, leader changed from <none> to 14aaf2229fb64d958dcd3c5a567b736b (127.8.18.193). New cstate: current_term: 1 leader_uuid: "14aaf2229fb64d958dcd3c5a567b736b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3516618266c8490dbd101f35d519fda1" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 40631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c9006673c2d6438b8f84388a08260f30" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 46139 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "14aaf2229fb64d958dcd3c5a567b736b" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 34541 } health_report { overall_health: HEALTHY } } }
W20260810 11:30:11.358366 9036 tablet.cc:2369] T 3251aee2986f46a4a7c1d97ab8993d85 P 14aaf2229fb64d958dcd3c5a567b736b: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 10 11:30:11 dist-test-slave-srgv krb5kdc[8801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361408, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.18.193@KRBTEST.COM
I20260810 11:30:11.376178 9325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:11.355216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45084 (local address 127.8.18.193:34541)
0810 11:30:11.355527 (+ 311us) server_negotiation.cc:206] Beginning negotiation
0810 11:30:11.355537 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 11:30:11.356384 (+ 847us) server_negotiation.cc:407] Connection header received
0810 11:30:11.356566 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:30:11.356570 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:30:11.356642 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:30:11.356745 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:30:11.359936 (+ 3191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:11.360769 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:11.362379 (+ 1610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:11.362857 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:11.362969 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:11.368571 (+ 5602us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:30:11.368589 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:30:11.368606 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:30:11.368656 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:30:11.372112 (+ 3456us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:11.373460 (+ 1348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:11.373482 (+ 22us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:11.373485 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:11.373702 (+ 217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:11.374298 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:11.374301 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:11.374303 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:11.374747 (+ 444us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:30:11.374996 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0810 11:30:11.375427 (+ 431us) server_negotiation.cc:299] Negotiation successful
0810 11:30:11.375709 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
Aug 10 11:30:11 dist-test-slave-srgv krb5kdc[8801](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361411, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 10 11:30:11 dist-test-slave-srgv krb5kdc[8801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361411, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.8.18.193@KRBTEST.COM
I20260810 11:30:11.436316 9325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:11.418528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45100 (local address 127.8.18.193:34541)
0810 11:30:11.418836 (+ 308us) server_negotiation.cc:206] Beginning negotiation
0810 11:30:11.418843 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:30:11.419110 (+ 267us) server_negotiation.cc:407] Connection header received
0810 11:30:11.419303 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:30:11.419309 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:30:11.419407 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:30:11.419522 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:30:11.421997 (+ 2475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:11.422836 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:11.424227 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:11.424760 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:11.424869 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:11.430037 (+ 5168us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:30:11.430043 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:30:11.430046 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:30:11.430095 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:30:11.433128 (+ 3033us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:11.434231 (+ 1103us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:11.434236 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:11.434238 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:11.434458 (+ 220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:11.434928 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:11.434932 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:11.434934 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:11.435226 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:30:11.435470 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0810 11:30:11.435765 (+ 295us) server_negotiation.cc:299] Negotiation successful
0810 11:30:11.435960 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118}
W20260810 11:30:11.443660 9172 tablet.cc:2369] T 3251aee2986f46a4a7c1d97ab8993d85 P c9006673c2d6438b8f84388a08260f30: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 10 11:30:11 dist-test-slave-srgv krb5kdc[8801](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361411, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 10 11:30:11 dist-test-slave-srgv krb5kdc[8801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361411, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.18.193@KRBTEST.COM
I20260810 11:30:11.501838 9325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:11.482151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45114 (local address 127.8.18.193:34541)
0810 11:30:11.482423 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0810 11:30:11.482430 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:30:11.482757 (+ 327us) server_negotiation.cc:407] Connection header received
0810 11:30:11.482968 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:30:11.482972 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:30:11.483059 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:30:11.483155 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:30:11.485175 (+ 2020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:11.486122 (+ 947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:11.487808 (+ 1686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:11.488287 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:11.488405 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:11.494501 (+ 6096us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:30:11.494507 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:30:11.494511 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:30:11.494559 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:30:11.498578 (+ 4019us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:11.499593 (+ 1015us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:11.499598 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:11.499601 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:11.499773 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:11.500238 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:11.500242 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:11.500244 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:11.500574 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:30:11.500808 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0810 11:30:11.501077 (+ 269us) server_negotiation.cc:299] Negotiation successful
0810 11:30:11.501267 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
W20260810 11:30:11.504215 8950 tablet_service.cc:2908] Rejecting scan request for tablet 3251aee2986f46a4a7c1d97ab8993d85: Uninitialized: clean time has not yet been initialized
I20260810 11:30:11.509364 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 8907
I20260810 11:30:11.525079 9325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:11.510660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45124 (local address 127.8.18.193:34541)
0810 11:30:11.510900 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0810 11:30:11.510907 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:30:11.511217 (+ 310us) server_negotiation.cc:407] Connection header received
0810 11:30:11.511322 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:30:11.511326 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:30:11.511412 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:30:11.511505 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:30:11.512961 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:11.513872 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:11.515333 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:11.515822 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:11.515926 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:11.518828 (+ 2902us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:30:11.518833 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:30:11.518837 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:30:11.518876 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:30:11.521882 (+ 3006us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:11.522732 (+ 850us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:11.522737 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:11.522739 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:11.522889 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:11.523305 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:11.523310 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:11.523313 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:11.523663 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:30:11.523894 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0810 11:30:11.524230 (+ 336us) server_negotiation.cc:299] Negotiation successful
0810 11:30:11.524424 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260810 11:30:11.526023 9030 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:30:11.626551 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 8907
I20260810 11:30:11.646677 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 9041
Aug 10 11:30:11 dist-test-slave-srgv krb5kdc[8801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361411, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.18.194@KRBTEST.COM
I20260810 11:30:11.663630 9323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:11.647530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46654 (local address 127.8.18.194:46139)
0810 11:30:11.647715 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0810 11:30:11.647720 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:30:11.647740 (+ 20us) server_negotiation.cc:407] Connection header received
0810 11:30:11.647872 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:30:11.647876 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:30:11.647982 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:30:11.648073 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:30:11.649384 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:11.650259 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:11.651704 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:11.652122 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:11.652230 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:11.657012 (+ 4782us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:30:11.657032 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:30:11.657051 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:30:11.657139 (+ 88us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:30:11.660240 (+ 3101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:11.661277 (+ 1037us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:11.661285 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:11.661287 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:11.661625 (+ 338us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:11.662064 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:11.662067 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:11.662069 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:11.662504 (+ 435us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:30:11.662754 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0810 11:30:11.663166 (+ 412us) server_negotiation.cc:299] Negotiation successful
0810 11:30:11.663356 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260810 11:30:11.664381 9166 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:30:11.757431 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 9041
I20260810 11:30:11.775900 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 9177
Aug 10 11:30:11 dist-test-slave-srgv krb5kdc[8801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361411, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.18.195@KRBTEST.COM
I20260810 11:30:11.792870 9321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:11.776892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40390 (local address 127.8.18.195:40631)
0810 11:30:11.777120 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0810 11:30:11.777124 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 11:30:11.777145 (+ 21us) server_negotiation.cc:407] Connection header received
0810 11:30:11.777496 (+ 351us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:30:11.777502 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:30:11.777603 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:30:11.777701 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:30:11.779012 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:11.779788 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:11.781230 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:11.781911 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:11.782020 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:11.786637 (+ 4617us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:30:11.786655 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:30:11.786672 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:30:11.786715 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:30:11.789799 (+ 3084us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:11.790600 (+ 801us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:11.790611 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:11.790614 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:11.790762 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:11.791184 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:11.791192 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:11.791197 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:11.791591 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:30:11.791891 (+ 300us) server_negotiation.cc:1035] Waiting for connection context
0810 11:30:11.792242 (+ 351us) server_negotiation.cc:299] Negotiation successful
0810 11:30:11.792481 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260810 11:30:11.793849 9302 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:30:11.885046 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 9177
I20260810 11:30:11.902791 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 8814
I20260810 11:30:11.904224 8876 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:30:12.021971 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 8814
2026-08-10T11:30:12Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (4703 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786361401922362-8267-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 11:30:12 dist-test-slave-srgv krb5kdc[9356](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 11:30:12 dist-test-slave-srgv krb5kdc[9356](info): set up 2 sockets
Aug 10 11:30:12 dist-test-slave-srgv krb5kdc[9356](info): commencing operation
krb5kdc: starting...
W20260810 11:30:13.144904 8267 mini_kdc.cc:121] Time spent starting KDC: real 1.077s user 0.000s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786361401922362-8267-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786361401922362-8267-0/krb5kdc/test-user.keytab.
Aug 10 11:30:13 dist-test-slave-srgv krb5kdc[9356](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361413, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T11:30:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T11:30:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.18.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.18.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.18.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.254.keytab.
Entry for principal kudu/127.8.18.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.254.keytab.
I20260810 11:30:13.369658 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786361401922362-8267-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786361401922362-8267-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786361401922362-8267-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.18.254:35405
--webserver_interface=127.8.18.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.254.keytab
--principal=kudu/127.8.18.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.18.212:38491
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.18.254:35405
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786361401922362-8267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786361401922362-8267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786361401922362-8267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:30:13.718998 9369 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:30:13.719350 9369 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:30:13.719447 9369 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:30:13.727722 9369 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 11:30:13.727867 9369 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260810 11:30:13.727946 9369 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:30:13.728101 9369 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:30:13.728196 9369 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 11:30:13.728307 9369 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 11:30:13.739295 9369 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.18.212:38491
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786361401922362-8267-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786361401922362-8267-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.18.254:35405
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.254.keytab
--principal=kudu/127.8.18.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.18.254:35405
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.18.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786361401922362-8267-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:30:13.740877 9369 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:30:13.742417 9369 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:30:13.755424 9375 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:30:13.755542 9374 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:30:13.755466 9377 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:30:13.756551 9369 server_base.cc:1034] running on GCE node
I20260810 11:30:13.757546 9369 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:30:13.759116 9369 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:30:13.760416 9369 hybrid_clock.cc:648] HybridClock initialized: now 1786361413760342 us; error 64 us; skew 500 ppm
I20260810 11:30:13.760780 9369 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:30:13 dist-test-slave-srgv krb5kdc[9356](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361413, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:30:13.768875 9369 init.cc:377] Logged in from keytab as kudu/127.8.18.254@KRBTEST.COM (short username kudu)
I20260810 11:30:13.770834 9369 webserver.cc:460] Webserver started at http://127.8.18.254:42689/ using document root <none> and password file <none>
I20260810 11:30:13.771646 9369 fs_manager.cc:359] Metadata directory not provided
I20260810 11:30:13.771798 9369 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:30:13.772174 9369 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:30:13.775676 9369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786361401922362-8267-0/minicluster-data/master-0/data/instance:
uuid: "41a39e4b02e143cda20aa5740d974b27"
format_stamp: "Formatted at 2026-08-10 11:30:13 on dist-test-slave-srgv"
server_key: "d554706c9fb1fb04c93d001dc821d546"
server_key_iv: "d8f656a7f6f3f30a5df7176f482ed828"
server_key_version: "encryptionkey@0"
I20260810 11:30:13.776630 9369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786361401922362-8267-0/minicluster-data/master-0/wal/instance:
uuid: "41a39e4b02e143cda20aa5740d974b27"
format_stamp: "Formatted at 2026-08-10 11:30:13 on dist-test-slave-srgv"
server_key: "d554706c9fb1fb04c93d001dc821d546"
server_key_iv: "d8f656a7f6f3f30a5df7176f482ed828"
server_key_version: "encryptionkey@0"
I20260810 11:30:13.782866 9369 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260810 11:30:13.787117 9384 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:30:13.788719 9369 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 11:30:13.788956 9369 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786361401922362-8267-0/minicluster-data/master-0/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786361401922362-8267-0/minicluster-data/master-0/wal
uuid: "41a39e4b02e143cda20aa5740d974b27"
format_stamp: "Formatted at 2026-08-10 11:30:13 on dist-test-slave-srgv"
server_key: "d554706c9fb1fb04c93d001dc821d546"
server_key_iv: "d8f656a7f6f3f30a5df7176f482ed828"
server_key_version: "encryptionkey@0"
I20260810 11:30:13.789276 9369 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786361401922362-8267-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786361401922362-8267-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786361401922362-8267-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:30:13.805610 9369 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:30:13.810947 9369 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:30:13.811286 9369 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:30:13.836757 9369 rpc_server.cc:307] RPC server started. Bound to: 127.8.18.254:35405
I20260810 11:30:13.836783 9446 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.18.254:35405 every 8 connection(s)
I20260810 11:30:13.838361 9369 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
I20260810 11:30:13.842631 9447 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:30:13.842648 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 9369
I20260810 11:30:13.842878 8267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786361401922362-8267-0/minicluster-data/master-0/wal/instance
I20260810 11:30:13.843614 8267 external_mini_cluster.cc:1428] Setting key ff7e5a46b59bd12ee3172a37e20bff6c
I20260810 11:30:13.858417 9447 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 41a39e4b02e143cda20aa5740d974b27: Bootstrap starting.
Aug 10 11:30:13 dist-test-slave-srgv krb5kdc[9356](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361413, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.18.254@KRBTEST.COM
I20260810 11:30:13.862005 9447 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 41a39e4b02e143cda20aa5740d974b27: Neither blocks nor log segments found. Creating new log.
I20260810 11:30:13.863538 9447 log.cc:824] T 00000000000000000000000000000000 P 41a39e4b02e143cda20aa5740d974b27: Log is configured to *not* fsync() on all Append() calls
I20260810 11:30:13.867594 9447 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 41a39e4b02e143cda20aa5740d974b27: No bootstrap required, opened a new log
I20260810 11:30:13.871975 9450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:13.845904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48560 (local address 127.8.18.254:35405)
0810 11:30:13.846814 (+ 910us) server_negotiation.cc:206] Beginning negotiation
0810 11:30:13.846850 (+ 36us) server_negotiation.cc:399] Waiting for connection header
0810 11:30:13.846911 (+ 61us) server_negotiation.cc:407] Connection header received
0810 11:30:13.849949 (+ 3038us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:30:13.849987 (+ 38us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:30:13.850653 (+ 666us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:30:13.851205 (+ 552us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:30:13.852702 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:13.854879 (+ 2177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:13.856259 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:13.857464 (+ 1205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:13.857672 (+ 208us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:13.862021 (+ 4349us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:30:13.862038 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:30:13.862071 (+ 33us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:30:13.862137 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:30:13.866304 (+ 4167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:13.867232 (+ 928us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:13.867242 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:13.867253 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:13.867500 (+ 247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:13.868155 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:13.868165 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:13.868170 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:13.868942 (+ 772us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:30:13.869330 (+ 388us) server_negotiation.cc:1035] Waiting for connection context
0810 11:30:13.869875 (+ 545us) server_negotiation.cc:299] Negotiation successful
0810 11:30:13.870421 (+ 546us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":650,"thread_start_us":387,"threads_started":1}
I20260810 11:30:13.877347 9447 raft_consensus.cc:348] T 00000000000000000000000000000000 P 41a39e4b02e143cda20aa5740d974b27 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41a39e4b02e143cda20aa5740d974b27" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 35405 } }
I20260810 11:30:13.877885 9447 raft_consensus.cc:374] T 00000000000000000000000000000000 P 41a39e4b02e143cda20aa5740d974b27 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:30:13.878012 9447 raft_consensus.cc:729] T 00000000000000000000000000000000 P 41a39e4b02e143cda20aa5740d974b27 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41a39e4b02e143cda20aa5740d974b27, State: Initialized, Role: FOLLOWER
I20260810 11:30:13.878639 9447 consensus_queue.cc:260] T 00000000000000000000000000000000 P 41a39e4b02e143cda20aa5740d974b27 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41a39e4b02e143cda20aa5740d974b27" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 35405 } }
I20260810 11:30:13.878929 9447 raft_consensus.cc:388] T 00000000000000000000000000000000 P 41a39e4b02e143cda20aa5740d974b27 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 11:30:13.879081 9447 raft_consensus.cc:482] T 00000000000000000000000000000000 P 41a39e4b02e143cda20aa5740d974b27 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 11:30:13.879268 9447 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 41a39e4b02e143cda20aa5740d974b27 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:30:13.881628 9447 raft_consensus.cc:504] T 00000000000000000000000000000000 P 41a39e4b02e143cda20aa5740d974b27 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41a39e4b02e143cda20aa5740d974b27" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 35405 } }
I20260810 11:30:13.882145 9447 leader_election.cc:302] T 00000000000000000000000000000000 P 41a39e4b02e143cda20aa5740d974b27 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 41a39e4b02e143cda20aa5740d974b27; no voters:
I20260810 11:30:13.882838 9447 leader_election.cc:288] T 00000000000000000000000000000000 P 41a39e4b02e143cda20aa5740d974b27 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 11:30:13.882961 9452 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 41a39e4b02e143cda20aa5740d974b27 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:30:13.884272 9452 raft_consensus.cc:686] T 00000000000000000000000000000000 P 41a39e4b02e143cda20aa5740d974b27 [term 1 LEADER]: Becoming Leader. State: Replica: 41a39e4b02e143cda20aa5740d974b27, State: Running, Role: LEADER
I20260810 11:30:13.884859 9452 consensus_queue.cc:237] T 00000000000000000000000000000000 P 41a39e4b02e143cda20aa5740d974b27 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41a39e4b02e143cda20aa5740d974b27" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 35405 } }
I20260810 11:30:13.885473 9447 sys_catalog.cc:564] T 00000000000000000000000000000000 P 41a39e4b02e143cda20aa5740d974b27 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 11:30:13.888495 9453 sys_catalog.cc:455] T 00000000000000000000000000000000 P 41a39e4b02e143cda20aa5740d974b27 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "41a39e4b02e143cda20aa5740d974b27" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41a39e4b02e143cda20aa5740d974b27" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 35405 } } }
I20260810 11:30:13.889004 9453 sys_catalog.cc:458] T 00000000000000000000000000000000 P 41a39e4b02e143cda20aa5740d974b27 [sys.catalog]: This master's current role is: LEADER
I20260810 11:30:13.889454 9454 sys_catalog.cc:455] T 00000000000000000000000000000000 P 41a39e4b02e143cda20aa5740d974b27 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 41a39e4b02e143cda20aa5740d974b27. Latest consensus state: current_term: 1 leader_uuid: "41a39e4b02e143cda20aa5740d974b27" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41a39e4b02e143cda20aa5740d974b27" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 35405 } } }
I20260810 11:30:13.889807 9454 sys_catalog.cc:458] T 00000000000000000000000000000000 P 41a39e4b02e143cda20aa5740d974b27 [sys.catalog]: This master's current role is: LEADER
I20260810 11:30:13.890764 9461 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 11:30:13.896781 9461 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 11:30:13.909092 9461 catalog_manager.cc:1347] Generated new cluster ID: 3493b8aafb8a4e608fa549b4ce38d5ba
I20260810 11:30:13.909248 9461 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 11:30:13.919816 9461 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 11:30:13.921469 9461 catalog_manager.cc:1504] Loading token signing keys...
I20260810 11:30:13.933341 9461 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 41a39e4b02e143cda20aa5740d974b27: Generated new TSK 0
I20260810 11:30:13.934582 9461 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.18.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.18.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.18.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.193.keytab.
Entry for principal kudu/127.8.18.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.193.keytab.
I20260810 11:30:13.993940 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786361401922362-8267-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786361401922362-8267-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786361401922362-8267-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786361401922362-8267-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.18.193:0
--local_ip_for_outbound_sockets=127.8.18.193
--webserver_interface=127.8.18.193
--webserver_port=0
--tserver_master_addrs=127.8.18.254:35405
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.193.keytab
--principal=kudu/127.8.18.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.18.212:38491
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786361401922362-8267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786361401922362-8267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786361401922362-8267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:30:14.351483 9474 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:30:14.351787 9474 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:30:14.351966 9474 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:30:14.360001 9474 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:30:14.360178 9474 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:30:14.360410 9474 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 11:30:14.360551 9474 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.18.193
I20260810 11:30:14.372748 9474 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.18.212:38491
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786361401922362-8267-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786361401922362-8267-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.193.keytab
--principal=kudu/127.8.18.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.18.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786361401922362-8267-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.18.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.18.254:35405
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.18.193
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786361401922362-8267-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:30:14.374287 9474 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:30:14.375876 9474 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:30:14.391316 9482 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:30:14.391302 9479 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:30:14.391583 9480 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:30:14.392338 9474 server_base.cc:1034] running on GCE node
I20260810 11:30:14.393074 9474 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:30:14.394407 9474 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:30:14.395676 9474 hybrid_clock.cc:648] HybridClock initialized: now 1786361414395591 us; error 61 us; skew 500 ppm
I20260810 11:30:14.395994 9474 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:30:14 dist-test-slave-srgv krb5kdc[9356](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361414, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:30:14.399856 9474 init.cc:377] Logged in from keytab as kudu/127.8.18.193@KRBTEST.COM (short username kudu)
I20260810 11:30:14.401778 9474 webserver.cc:460] Webserver started at http://127.8.18.193:40635/ using document root <none> and password file <none>
I20260810 11:30:14.402623 9474 fs_manager.cc:359] Metadata directory not provided
I20260810 11:30:14.402727 9474 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:30:14.403106 9474 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:30:14.406531 9474 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786361401922362-8267-0/minicluster-data/ts-0/data/instance:
uuid: "c88a7a11405f4a7699f9573f7dcfd4c0"
format_stamp: "Formatted at 2026-08-10 11:30:14 on dist-test-slave-srgv"
server_key: "27ff2449f0a680c5acb126b5e2994c7c"
server_key_iv: "1878f3e624ad521e9d4f37e3e694abc0"
server_key_version: "encryptionkey@0"
I20260810 11:30:14.407500 9474 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786361401922362-8267-0/minicluster-data/ts-0/wal/instance:
uuid: "c88a7a11405f4a7699f9573f7dcfd4c0"
format_stamp: "Formatted at 2026-08-10 11:30:14 on dist-test-slave-srgv"
server_key: "27ff2449f0a680c5acb126b5e2994c7c"
server_key_iv: "1878f3e624ad521e9d4f37e3e694abc0"
server_key_version: "encryptionkey@0"
I20260810 11:30:14.413832 9474 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260810 11:30:14.418031 9489 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:30:14.419665 9474 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 11:30:14.419895 9474 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786361401922362-8267-0/minicluster-data/ts-0/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786361401922362-8267-0/minicluster-data/ts-0/wal
uuid: "c88a7a11405f4a7699f9573f7dcfd4c0"
format_stamp: "Formatted at 2026-08-10 11:30:14 on dist-test-slave-srgv"
server_key: "27ff2449f0a680c5acb126b5e2994c7c"
server_key_iv: "1878f3e624ad521e9d4f37e3e694abc0"
server_key_version: "encryptionkey@0"
I20260810 11:30:14.420181 9474 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786361401922362-8267-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786361401922362-8267-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786361401922362-8267-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:30:14.446238 9474 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:30:14.451538 9474 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:30:14.451874 9474 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:30:14.455142 9474 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:30:14.455274 9474 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:30:14.455427 9474 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:30:14.455508 9474 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 11:30:14 dist-test-slave-srgv krb5kdc[9356](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361414, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.193@KRBTEST.COM for kudu/127.8.18.254@KRBTEST.COM
I20260810 11:30:14.482087 9498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:14.455844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.193:56141 (local address 127.8.18.254:35405)
0810 11:30:14.456367 (+ 523us) server_negotiation.cc:206] Beginning negotiation
0810 11:30:14.456375 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:30:14.458067 (+ 1692us) server_negotiation.cc:407] Connection header received
0810 11:30:14.460377 (+ 2310us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:30:14.460384 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:30:14.460503 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:30:14.460657 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:30:14.463318 (+ 2661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:14.464485 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:14.467627 (+ 3142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:14.468072 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:14.468247 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:14.474232 (+ 5985us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:30:14.474245 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:30:14.474254 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:30:14.474318 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:30:14.478459 (+ 4141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:14.479382 (+ 923us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:14.479389 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:14.479393 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:14.479541 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:14.480062 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:14.480066 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:14.480068 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:14.480348 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:30:14.480534 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0810 11:30:14.481381 (+ 847us) server_negotiation.cc:299] Negotiation successful
0810 11:30:14.481676 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":380,"thread_start_us":1163,"threads_started":1}
I20260810 11:30:14.483649 9499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:14.456835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:35405 (local address 127.8.18.193:56141)
0810 11:30:14.457862 (+ 1027us) negotiation.cc:107] Waiting for socket to connect
0810 11:30:14.457918 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0810 11:30:14.460006 (+ 2088us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:30:14.460944 (+ 938us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:30:14.460953 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:30:14.461875 (+ 922us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:30:14.463040 (+ 1165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:30:14.463099 (+ 59us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:30:14.464727 (+ 1628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:30:14.464731 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:30:14.467368 (+ 2637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:30:14.467394 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:30:14.468389 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:30:14.468398 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:30:14.468649 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:14.469797 (+ 1148us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:30:14.469843 (+ 46us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:30:14.473773 (+ 3930us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:30:14.478660 (+ 4887us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:30:14.478671 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:30:14.478692 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:30:14.479213 (+ 521us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:30:14.479667 (+ 454us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:30:14.479673 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:30:14.479676 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:30:14.479907 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:30:14.480494 (+ 587us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:30:14.480511 (+ 17us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:30:14.480941 (+ 430us) client_negotiation.cc:769] Sending connection context
0810 11:30:14.481241 (+ 300us) client_negotiation.cc:240] Negotiation successful
0810 11:30:14.481923 (+ 682us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":741,"thread_start_us":469,"threads_started":1}
I20260810 11:30:14.506976 9474 rpc_server.cc:307] RPC server started. Bound to: 127.8.18.193:39295
I20260810 11:30:14.507025 9610 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.18.193:39295 every 8 connection(s)
I20260810 11:30:14.508497 9474 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786361401922362-8267-0/minicluster-data/ts-0/data/info.pb
I20260810 11:30:14.510205 9498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:14.493872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.193:44295 (local address 127.8.18.254:35405)
0810 11:30:14.494122 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0810 11:30:14.494131 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 11:30:14.495007 (+ 876us) server_negotiation.cc:407] Connection header received
0810 11:30:14.495281 (+ 274us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:30:14.495288 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:30:14.495367 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:30:14.495481 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:30:14.497103 (+ 1622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:14.497969 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:14.499592 (+ 1623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:14.500320 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:14.500520 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:14.503764 (+ 3244us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:30:14.503769 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:30:14.503773 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:30:14.503813 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:30:14.507035 (+ 3222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:14.507858 (+ 823us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:14.507865 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:14.507868 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:14.508012 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:14.508451 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:14.508455 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:14.508457 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:14.508826 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:30:14.508991 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0810 11:30:14.509565 (+ 574us) server_negotiation.cc:299] Negotiation successful
0810 11:30:14.509820 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260810 11:30:14.510424 9579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:14.493875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:35405 (local address 127.8.18.193:44295)
0810 11:30:14.494829 (+ 954us) negotiation.cc:107] Waiting for socket to connect
0810 11:30:14.494865 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0810 11:30:14.495075 (+ 210us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:30:14.495514 (+ 439us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:30:14.495519 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:30:14.496401 (+ 882us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:30:14.496920 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:30:14.496937 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:30:14.498176 (+ 1239us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:30:14.498179 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:30:14.499411 (+ 1232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:30:14.499430 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:30:14.500843 (+ 1413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:30:14.500848 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:30:14.501145 (+ 297us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:14.502133 (+ 988us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:30:14.502169 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:30:14.503499 (+ 1330us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:30:14.507202 (+ 3703us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:30:14.507208 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:30:14.507213 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:30:14.507705 (+ 492us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:30:14.508154 (+ 449us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:30:14.508160 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:30:14.508164 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:30:14.508303 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:30:14.508951 (+ 648us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:30:14.508957 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:30:14.509172 (+ 215us) client_negotiation.cc:769] Sending connection context
0810 11:30:14.509475 (+ 303us) client_negotiation.cc:240] Negotiation successful
0810 11:30:14.509887 (+ 412us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":779,"thread_start_us":596,"threads_started":1}
I20260810 11:30:14.511807 9611 heartbeater.cc:347] Connected to a master server at 127.8.18.254:35405
I20260810 11:30:14.512593 9611 heartbeater.cc:464] Registering TS with master...
I20260810 11:30:14.513465 9611 heartbeater.cc:511] Master 127.8.18.254:35405 requested a full tablet report, sending...
I20260810 11:30:14.515890 9401 ts_manager.cc:194] Registered new tserver with Master: c88a7a11405f4a7699f9573f7dcfd4c0 (127.8.18.193:39295)
I20260810 11:30:14.517588 9401 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.18.193@KRBTEST.COM'} at 127.8.18.193:56141
I20260810 11:30:14.518914 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 9474
I20260810 11:30:14.519176 8267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786361401922362-8267-0/minicluster-data/ts-0/wal/instance
I20260810 11:30:14.520088 8267 external_mini_cluster.cc:1428] Setting key 0dd50e63da8caaef869b0c9fc8b36656
WARNING: no policy specified for kudu/127.8.18.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.18.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.18.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.194.keytab.
Entry for principal kudu/127.8.18.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.194.keytab.
I20260810 11:30:14.569939 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786361401922362-8267-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786361401922362-8267-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786361401922362-8267-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786361401922362-8267-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.18.194:0
--local_ip_for_outbound_sockets=127.8.18.194
--webserver_interface=127.8.18.194
--webserver_port=0
--tserver_master_addrs=127.8.18.254:35405
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.194.keytab
--principal=kudu/127.8.18.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.18.212:38491
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786361401922362-8267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786361401922362-8267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786361401922362-8267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:30:14.927502 9615 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:30:14.927848 9615 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:30:14.927953 9615 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:30:14.935794 9615 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:30:14.935943 9615 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:30:14.936033 9615 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 11:30:14.936116 9615 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.18.194
I20260810 11:30:14.946951 9615 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.18.212:38491
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786361401922362-8267-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786361401922362-8267-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.194.keytab
--principal=kudu/127.8.18.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.18.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786361401922362-8267-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.18.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.18.254:35405
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.18.194
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786361401922362-8267-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:30:14.948340 9615 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:30:14.949831 9615 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:30:14.964700 9620 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:30:14.964794 9621 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:30:14.964812 9623 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:30:14.965833 9615 server_base.cc:1034] running on GCE node
I20260810 11:30:14.966683 9615 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:30:14.967876 9615 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:30:14.969204 9615 hybrid_clock.cc:648] HybridClock initialized: now 1786361414969138 us; error 94 us; skew 500 ppm
I20260810 11:30:14.969681 9615 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:30:14 dist-test-slave-srgv krb5kdc[9356](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361414, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:30:14.973635 9615 init.cc:377] Logged in from keytab as kudu/127.8.18.194@KRBTEST.COM (short username kudu)
I20260810 11:30:14.975484 9615 webserver.cc:460] Webserver started at http://127.8.18.194:37865/ using document root <none> and password file <none>
I20260810 11:30:14.976414 9615 fs_manager.cc:359] Metadata directory not provided
I20260810 11:30:14.976575 9615 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:30:14.976959 9615 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:30:14.980315 9615 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786361401922362-8267-0/minicluster-data/ts-1/data/instance:
uuid: "bb46349f489247b4973643a309634d45"
format_stamp: "Formatted at 2026-08-10 11:30:14 on dist-test-slave-srgv"
server_key: "37cfeeb01db927cdad6127a7881bd276"
server_key_iv: "ce20009aa75234598c02e9c82acbe9a4"
server_key_version: "encryptionkey@0"
I20260810 11:30:14.981276 9615 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786361401922362-8267-0/minicluster-data/ts-1/wal/instance:
uuid: "bb46349f489247b4973643a309634d45"
format_stamp: "Formatted at 2026-08-10 11:30:14 on dist-test-slave-srgv"
server_key: "37cfeeb01db927cdad6127a7881bd276"
server_key_iv: "ce20009aa75234598c02e9c82acbe9a4"
server_key_version: "encryptionkey@0"
I20260810 11:30:14.987498 9615 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260810 11:30:14.991968 9630 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:30:14.993557 9615 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 11:30:14.993875 9615 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786361401922362-8267-0/minicluster-data/ts-1/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786361401922362-8267-0/minicluster-data/ts-1/wal
uuid: "bb46349f489247b4973643a309634d45"
format_stamp: "Formatted at 2026-08-10 11:30:14 on dist-test-slave-srgv"
server_key: "37cfeeb01db927cdad6127a7881bd276"
server_key_iv: "ce20009aa75234598c02e9c82acbe9a4"
server_key_version: "encryptionkey@0"
I20260810 11:30:14.994139 9615 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786361401922362-8267-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786361401922362-8267-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786361401922362-8267-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:30:15.012962 9615 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:30:15.018620 9615 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:30:15.018973 9615 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:30:15.022177 9615 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:30:15.022327 9615 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:30:15.022528 9615 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:30:15.022619 9615 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 11:30:15 dist-test-slave-srgv krb5kdc[9356](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361414, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.194@KRBTEST.COM for kudu/127.8.18.254@KRBTEST.COM
I20260810 11:30:15.052250 9639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:15.023435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.194:44749 (local address 127.8.18.254:35405)
0810 11:30:15.024221 (+ 786us) server_negotiation.cc:206] Beginning negotiation
0810 11:30:15.024228 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:30:15.026353 (+ 2125us) server_negotiation.cc:407] Connection header received
0810 11:30:15.029923 (+ 3570us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:30:15.029931 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:30:15.030083 (+ 152us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:30:15.030252 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:30:15.033027 (+ 2775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:15.034488 (+ 1461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:15.037071 (+ 2583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:15.037770 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:15.037950 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:15.044285 (+ 6335us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:30:15.044313 (+ 28us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:30:15.044319 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:30:15.044378 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:30:15.048512 (+ 4134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:15.049466 (+ 954us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:15.049474 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:15.049482 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:15.049621 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:15.050173 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:15.050180 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:15.050183 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:15.050482 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:30:15.050660 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0810 11:30:15.051494 (+ 834us) server_negotiation.cc:299] Negotiation successful
0810 11:30:15.051816 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":517,"thread_start_us":299,"threads_started":1}
I20260810 11:30:15.053685 9640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:15.024932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:35405 (local address 127.8.18.194:44749)
0810 11:30:15.026076 (+ 1144us) negotiation.cc:107] Waiting for socket to connect
0810 11:30:15.026151 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0810 11:30:15.029368 (+ 3217us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:30:15.030581 (+ 1213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:30:15.030590 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:30:15.031622 (+ 1032us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:30:15.032699 (+ 1077us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:30:15.032743 (+ 44us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:30:15.034715 (+ 1972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:30:15.034723 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:30:15.036776 (+ 2053us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:30:15.036805 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:30:15.038119 (+ 1314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:30:15.038129 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:30:15.038610 (+ 481us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:15.039819 (+ 1209us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:30:15.039847 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:30:15.043908 (+ 4061us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:30:15.048730 (+ 4822us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:30:15.048742 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:30:15.048775 (+ 33us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:30:15.049274 (+ 499us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:30:15.049778 (+ 504us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:30:15.049784 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:30:15.049787 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:30:15.050040 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:30:15.050638 (+ 598us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:30:15.050648 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:30:15.051084 (+ 436us) client_negotiation.cc:769] Sending connection context
0810 11:30:15.051409 (+ 325us) client_negotiation.cc:240] Negotiation successful
0810 11:30:15.052084 (+ 675us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":788,"thread_start_us":525,"threads_started":1}
I20260810 11:30:15.080279 9639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:15.062458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.194:36809 (local address 127.8.18.254:35405)
0810 11:30:15.062754 (+ 296us) server_negotiation.cc:206] Beginning negotiation
0810 11:30:15.062764 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 11:30:15.063433 (+ 669us) server_negotiation.cc:407] Connection header received
0810 11:30:15.063685 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:30:15.063692 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:30:15.063818 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:30:15.063972 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:30:15.065105 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:15.066576 (+ 1471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:15.069165 (+ 2589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:15.069706 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:15.069866 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:15.072029 (+ 2163us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:30:15.072038 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:30:15.072046 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:30:15.072116 (+ 70us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:30:15.076560 (+ 4444us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:15.077841 (+ 1281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:15.077846 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:15.077849 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:15.078044 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:15.078595 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:15.078599 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:15.078601 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:15.078952 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:30:15.079153 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0810 11:30:15.079802 (+ 649us) server_negotiation.cc:299] Negotiation successful
0810 11:30:15.080008 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260810 11:30:15.080928 9717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:15.062507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:35405 (local address 127.8.18.194:36809)
0810 11:30:15.063281 (+ 774us) negotiation.cc:107] Waiting for socket to connect
0810 11:30:15.063306 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0810 11:30:15.063531 (+ 225us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:30:15.064028 (+ 497us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:30:15.064033 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:30:15.064597 (+ 564us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:30:15.064909 (+ 312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:30:15.064919 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:30:15.066805 (+ 1886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:30:15.066811 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:30:15.068847 (+ 2036us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:30:15.068877 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:30:15.069941 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:30:15.069948 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:30:15.070151 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:15.070879 (+ 728us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:30:15.070898 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:30:15.071753 (+ 855us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:30:15.076802 (+ 5049us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:30:15.076810 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:30:15.076820 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:30:15.077611 (+ 791us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:30:15.078256 (+ 645us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:30:15.078261 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:30:15.078264 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:30:15.078452 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:30:15.079129 (+ 677us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:30:15.079138 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:30:15.079466 (+ 328us) client_negotiation.cc:769] Sending connection context
0810 11:30:15.079692 (+ 226us) client_negotiation.cc:240] Negotiation successful
0810 11:30:15.080229 (+ 537us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":607,"thread_start_us":439,"threads_started":1}
I20260810 11:30:15.083094 9615 rpc_server.cc:307] RPC server started. Bound to: 127.8.18.194:44751
I20260810 11:30:15.083119 9751 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.18.194:44751 every 8 connection(s)
I20260810 11:30:15.084923 9615 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786361401922362-8267-0/minicluster-data/ts-1/data/info.pb
I20260810 11:30:15.087461 9752 heartbeater.cc:347] Connected to a master server at 127.8.18.254:35405
I20260810 11:30:15.087718 9752 heartbeater.cc:464] Registering TS with master...
I20260810 11:30:15.088455 9752 heartbeater.cc:511] Master 127.8.18.254:35405 requested a full tablet report, sending...
I20260810 11:30:15.090149 9401 ts_manager.cc:194] Registered new tserver with Master: bb46349f489247b4973643a309634d45 (127.8.18.194:44751)
I20260810 11:30:15.091050 9401 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.18.194@KRBTEST.COM'} at 127.8.18.194:44749
I20260810 11:30:15.094022 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 9615
I20260810 11:30:15.094246 8267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786361401922362-8267-0/minicluster-data/ts-1/wal/instance
I20260810 11:30:15.094703 8267 external_mini_cluster.cc:1428] Setting key 1de5c49a37930de7874b0d8da231f85c
WARNING: no policy specified for kudu/127.8.18.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.18.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.18.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.195.keytab.
Entry for principal kudu/127.8.18.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.195.keytab.
I20260810 11:30:15.146667 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786361401922362-8267-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786361401922362-8267-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786361401922362-8267-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786361401922362-8267-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.18.195:0
--local_ip_for_outbound_sockets=127.8.18.195
--webserver_interface=127.8.18.195
--webserver_port=0
--tserver_master_addrs=127.8.18.254:35405
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.195.keytab
--principal=kudu/127.8.18.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.18.212:38491
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786361401922362-8267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786361401922362-8267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786361401922362-8267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:30:15.503074 9756 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:30:15.503420 9756 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:30:15.503593 9756 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:30:15.511935 9756 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:30:15.512084 9756 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:30:15.512300 9756 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 11:30:15.512425 9756 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.18.195
I20260810 11:30:15.521914 9611 heartbeater.cc:503] Master 127.8.18.254:35405 was elected leader, sending a full tablet report...
I20260810 11:30:15.523872 9756 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.18.212:38491
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786361401922362-8267-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786361401922362-8267-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.195.keytab
--principal=kudu/127.8.18.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.18.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786361401922362-8267-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.18.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.18.254:35405
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.18.195
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786361401922362-8267-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:30:15.525573 9756 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:30:15.527067 9756 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:30:15.542063 9762 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:30:15.542014 9764 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:30:15.542090 9761 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:30:15.542999 9756 server_base.cc:1034] running on GCE node
I20260810 11:30:15.543686 9756 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:30:15.544819 9756 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:30:15.546139 9756 hybrid_clock.cc:648] HybridClock initialized: now 1786361415546056 us; error 56 us; skew 500 ppm
I20260810 11:30:15.546556 9756 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:30:15 dist-test-slave-srgv krb5kdc[9356](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361415, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:30:15.550323 9756 init.cc:377] Logged in from keytab as kudu/127.8.18.195@KRBTEST.COM (short username kudu)
I20260810 11:30:15.552120 9756 webserver.cc:460] Webserver started at http://127.8.18.195:42933/ using document root <none> and password file <none>
I20260810 11:30:15.552953 9756 fs_manager.cc:359] Metadata directory not provided
I20260810 11:30:15.553113 9756 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:30:15.553560 9756 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:30:15.557103 9756 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786361401922362-8267-0/minicluster-data/ts-2/data/instance:
uuid: "90566adbea5742dea2a7249d96470451"
format_stamp: "Formatted at 2026-08-10 11:30:15 on dist-test-slave-srgv"
server_key: "fb550a03858e2b98f3571b91c8cebf47"
server_key_iv: "088faca21978a6c12444ea01d51f89c7"
server_key_version: "encryptionkey@0"
I20260810 11:30:15.558171 9756 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786361401922362-8267-0/minicluster-data/ts-2/wal/instance:
uuid: "90566adbea5742dea2a7249d96470451"
format_stamp: "Formatted at 2026-08-10 11:30:15 on dist-test-slave-srgv"
server_key: "fb550a03858e2b98f3571b91c8cebf47"
server_key_iv: "088faca21978a6c12444ea01d51f89c7"
server_key_version: "encryptionkey@0"
I20260810 11:30:15.564396 9756 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260810 11:30:15.568737 9771 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:30:15.570592 9756 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260810 11:30:15.570804 9756 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786361401922362-8267-0/minicluster-data/ts-2/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786361401922362-8267-0/minicluster-data/ts-2/wal
uuid: "90566adbea5742dea2a7249d96470451"
format_stamp: "Formatted at 2026-08-10 11:30:15 on dist-test-slave-srgv"
server_key: "fb550a03858e2b98f3571b91c8cebf47"
server_key_iv: "088faca21978a6c12444ea01d51f89c7"
server_key_version: "encryptionkey@0"
I20260810 11:30:15.571043 9756 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786361401922362-8267-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786361401922362-8267-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786361401922362-8267-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:30:15.600168 9756 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:30:15.605664 9756 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:30:15.605986 9756 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:30:15.609068 9756 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:30:15.609171 9756 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:30:15.609454 9756 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:30:15.609560 9756 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
Aug 10 11:30:15 dist-test-slave-srgv krb5kdc[9356](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361415, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.195@KRBTEST.COM for kudu/127.8.18.254@KRBTEST.COM
I20260810 11:30:15.640981 9780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:15.610203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.195:50395 (local address 127.8.18.254:35405)
0810 11:30:15.610751 (+ 548us) server_negotiation.cc:206] Beginning negotiation
0810 11:30:15.610762 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 11:30:15.612399 (+ 1637us) server_negotiation.cc:407] Connection header received
0810 11:30:15.615416 (+ 3017us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:30:15.615426 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:30:15.615585 (+ 159us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:30:15.615748 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:30:15.618622 (+ 2874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:15.619785 (+ 1163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:15.622428 (+ 2643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:15.622897 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:15.623076 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:15.630419 (+ 7343us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:30:15.630431 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:30:15.630439 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:30:15.630502 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:30:15.635236 (+ 4734us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:15.636933 (+ 1697us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:15.636941 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:15.636946 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:15.637181 (+ 235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:15.638034 (+ 853us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:15.638042 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:15.638045 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:15.638470 (+ 425us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:30:15.638665 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0810 11:30:15.640094 (+ 1429us) server_negotiation.cc:299] Negotiation successful
0810 11:30:15.640407 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":404,"thread_start_us":249,"threads_started":1}
I20260810 11:30:15.643035 9781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:15.611003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:35405 (local address 127.8.18.195:50395)
0810 11:30:15.612102 (+ 1099us) negotiation.cc:107] Waiting for socket to connect
0810 11:30:15.612181 (+ 79us) client_negotiation.cc:174] Beginning negotiation
0810 11:30:15.614951 (+ 2770us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:30:15.616160 (+ 1209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:30:15.616180 (+ 20us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:30:15.617124 (+ 944us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:30:15.618335 (+ 1211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:30:15.618371 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:30:15.620109 (+ 1738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:30:15.620116 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:30:15.622199 (+ 2083us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:30:15.622219 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:30:15.623314 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:30:15.623324 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:30:15.623555 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:15.624631 (+ 1076us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:30:15.624662 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:30:15.629925 (+ 5263us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:30:15.635556 (+ 5631us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:30:15.635569 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:30:15.635596 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:30:15.636639 (+ 1043us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:30:15.637355 (+ 716us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:30:15.637364 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:30:15.637370 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:30:15.637840 (+ 470us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:30:15.638656 (+ 816us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:30:15.638673 (+ 17us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:30:15.639435 (+ 762us) client_negotiation.cc:769] Sending connection context
0810 11:30:15.639900 (+ 465us) client_negotiation.cc:240] Negotiation successful
0810 11:30:15.640739 (+ 839us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":742,"thread_start_us":489,"threads_started":1}
I20260810 11:30:15.654304 9756 rpc_server.cc:307] RPC server started. Bound to: 127.8.18.195:41353
I20260810 11:30:15.654366 9893 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.18.195:41353 every 8 connection(s)
I20260810 11:30:15.656101 9756 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786361401922362-8267-0/minicluster-data/ts-2/data/info.pb
I20260810 11:30:15.658708 9894 heartbeater.cc:347] Connected to a master server at 127.8.18.254:35405
I20260810 11:30:15.659015 9894 heartbeater.cc:464] Registering TS with master...
I20260810 11:30:15.659839 9894 heartbeater.cc:511] Master 127.8.18.254:35405 requested a full tablet report, sending...
I20260810 11:30:15.660969 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 9756
I20260810 11:30:15.661235 8267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786361401922362-8267-0/minicluster-data/ts-2/wal/instance
I20260810 11:30:15.661829 8267 external_mini_cluster.cc:1428] Setting key d17f2029afa401b2d97d31bbe2e4956d
I20260810 11:30:15.661793 9401 ts_manager.cc:194] Registered new tserver with Master: 90566adbea5742dea2a7249d96470451 (127.8.18.195:41353)
I20260810 11:30:15.662950 9401 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.18.195@KRBTEST.COM'} at 127.8.18.195:50395
I20260810 11:30:15.665390 8267 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 11:30:15.669016 9780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:15.652384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.195:59119 (local address 127.8.18.254:35405)
0810 11:30:15.652630 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0810 11:30:15.652640 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 11:30:15.653702 (+ 1062us) server_negotiation.cc:407] Connection header received
0810 11:30:15.653948 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:30:15.653952 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:30:15.654119 (+ 167us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:30:15.654247 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:30:15.655572 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:15.656522 (+ 950us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:15.659094 (+ 2572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:15.659580 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:15.659688 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:15.662034 (+ 2346us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:30:15.662039 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:30:15.662046 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:30:15.662081 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:30:15.666029 (+ 3948us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:15.666897 (+ 868us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:15.666903 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:15.666907 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:15.667054 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:15.667477 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:15.667482 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:15.667485 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:15.667782 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:30:15.667965 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0810 11:30:15.668373 (+ 408us) server_negotiation.cc:299] Negotiation successful
0810 11:30:15.668616 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260810 11:30:15.669068 9891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:15.652414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:35405 (local address 127.8.18.195:59119)
0810 11:30:15.653553 (+ 1139us) negotiation.cc:107] Waiting for socket to connect
0810 11:30:15.653575 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 11:30:15.653790 (+ 215us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:30:15.654317 (+ 527us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:30:15.654324 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:30:15.654974 (+ 650us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:30:15.655375 (+ 401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:30:15.655389 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:30:15.656740 (+ 1351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:30:15.656772 (+ 32us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:30:15.658909 (+ 2137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:30:15.658926 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:30:15.659803 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:30:15.659813 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:30:15.660016 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:15.660790 (+ 774us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:30:15.660815 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:30:15.661804 (+ 989us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:30:15.666231 (+ 4427us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:30:15.666236 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:30:15.666240 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:30:15.666741 (+ 501us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:30:15.667182 (+ 441us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:30:15.667187 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:30:15.667191 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:30:15.667344 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:30:15.667911 (+ 567us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:30:15.667916 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:30:15.668101 (+ 185us) client_negotiation.cc:769] Sending connection context
0810 11:30:15.668257 (+ 156us) client_negotiation.cc:240] Negotiation successful
0810 11:30:15.668614 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":932,"thread_start_us":790,"threads_started":1}
Aug 10 11:30:15 dist-test-slave-srgv krb5kdc[9356](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361415, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 10 11:30:15 dist-test-slave-srgv krb5kdc[9356](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361415, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.8.18.254@KRBTEST.COM
I20260810 11:30:15.723026 9780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:15.706092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48576 (local address 127.8.18.254:35405)
0810 11:30:15.706366 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0810 11:30:15.706374 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:30:15.706588 (+ 214us) server_negotiation.cc:407] Connection header received
0810 11:30:15.706870 (+ 282us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:30:15.706875 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:30:15.706946 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:30:15.707060 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:30:15.708671 (+ 1611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:15.709507 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:15.710902 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:15.711443 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:15.711597 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:15.716287 (+ 4690us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:30:15.716293 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:30:15.716297 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:30:15.716346 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:30:15.719471 (+ 3125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:15.720605 (+ 1134us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:15.720611 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:15.720613 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:15.720795 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:15.721295 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:15.721306 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:15.721309 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:15.721715 (+ 406us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:30:15.722021 (+ 306us) server_negotiation.cc:1035] Waiting for connection context
0810 11:30:15.722393 (+ 372us) server_negotiation.cc:299] Negotiation successful
0810 11:30:15.722665 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107}
I20260810 11:30:15.728353 9401 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48576:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 11:30:15.735770 9401 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 11:30:15.779237 9907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:15.750126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.195:41353 (local address 127.0.0.1:43176)
0810 11:30:15.766752 (+ 16626us) negotiation.cc:107] Waiting for socket to connect
0810 11:30:15.766818 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0810 11:30:15.768389 (+ 1571us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:30:15.771270 (+ 2881us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:30:15.771275 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:30:15.771301 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:30:15.771748 (+ 447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:30:15.771760 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:30:15.773904 (+ 2144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:30:15.773915 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:30:15.776454 (+ 2539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:30:15.776474 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:30:15.777976 (+ 1502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:30:15.777983 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:30:15.778325 (+ 342us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:15.778406 (+ 81us) client_negotiation.cc:769] Sending connection context
0810 11:30:15.778581 (+ 175us) client_negotiation.cc:240] Negotiation successful
0810 11:30:15.778808 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16199,"thread_start_us":15995,"threads_started":1}
I20260810 11:30:15.779368 9912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:15.751760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49340 (local address 127.8.18.193:39295)
0810 11:30:15.769184 (+ 17424us) server_negotiation.cc:206] Beginning negotiation
0810 11:30:15.769194 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 11:30:15.770077 (+ 883us) server_negotiation.cc:407] Connection header received
0810 11:30:15.770199 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:30:15.770207 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:30:15.770468 (+ 261us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:30:15.771322 (+ 854us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:30:15.771521 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:15.772718 (+ 1197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:15.776181 (+ 3463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:15.777800 (+ 1619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:15.777940 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:15.778169 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0810 11:30:15.778865 (+ 696us) server_negotiation.cc:299] Negotiation successful
0810 11:30:15.779059 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17244,"thread_start_us":17098,"threads_started":1}
I20260810 11:30:15.779364 9909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:15.750667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.194:44751 (local address 127.0.0.1:35768)
0810 11:30:15.768257 (+ 17590us) negotiation.cc:107] Waiting for socket to connect
0810 11:30:15.768281 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0810 11:30:15.768426 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:30:15.769737 (+ 1311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:30:15.769748 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:30:15.769785 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:30:15.770220 (+ 435us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:30:15.770233 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:30:15.773322 (+ 3089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:30:15.773328 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:30:15.775958 (+ 2630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:30:15.775968 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:30:15.778397 (+ 2429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:30:15.778402 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:30:15.778591 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:15.778657 (+ 66us) client_negotiation.cc:769] Sending connection context
0810 11:30:15.778743 (+ 86us) client_negotiation.cc:240] Negotiation successful
0810 11:30:15.778900 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17440,"mutex_wait_us":152,"thread_start_us":17250,"threads_started":1}
I20260810 11:30:15.779184 9908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:15.750444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43176 (local address 127.8.18.195:41353)
0810 11:30:15.770343 (+ 19899us) server_negotiation.cc:206] Beginning negotiation
0810 11:30:15.770352 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 11:30:15.770388 (+ 36us) server_negotiation.cc:407] Connection header received
0810 11:30:15.770501 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:30:15.770509 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:30:15.770736 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:30:15.770951 (+ 215us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:30:15.772024 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:15.773746 (+ 1722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:15.776650 (+ 2904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:15.777753 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:15.777923 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:15.778199 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0810 11:30:15.778644 (+ 445us) server_negotiation.cc:299] Negotiation successful
0810 11:30:15.778821 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19701,"thread_start_us":19518,"threads_started":1}
I20260810 11:30:15.780368 9910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:15.751002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.193:39295 (local address 127.0.0.1:49340)
0810 11:30:15.769770 (+ 18768us) negotiation.cc:107] Waiting for socket to connect
0810 11:30:15.769788 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 11:30:15.769924 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:30:15.770674 (+ 750us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:30:15.770679 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:30:15.770707 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:30:15.771109 (+ 402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:30:15.771124 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:30:15.772868 (+ 1744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:30:15.772876 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:30:15.775938 (+ 3062us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:30:15.775956 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:30:15.778383 (+ 2427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:30:15.778390 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:30:15.778647 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:15.778673 (+ 26us) client_negotiation.cc:769] Sending connection context
0810 11:30:15.778750 (+ 77us) client_negotiation.cc:240] Negotiation successful
0810 11:30:15.780039 (+ 1289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18580,"mutex_wait_us":1120,"thread_start_us":18437,"threads_started":1}
I20260810 11:30:15.782303 9911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:15.751413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35768 (local address 127.8.18.194:44751)
0810 11:30:15.767667 (+ 16254us) server_negotiation.cc:206] Beginning negotiation
0810 11:30:15.767681 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 11:30:15.768663 (+ 982us) server_negotiation.cc:407] Connection header received
0810 11:30:15.768804 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:30:15.768817 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:30:15.769145 (+ 328us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:30:15.769432 (+ 287us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:30:15.770415 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:15.773158 (+ 2743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:15.776180 (+ 3022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:15.777801 (+ 1621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:15.777940 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:15.778169 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0810 11:30:15.781769 (+ 3600us) server_negotiation.cc:299] Negotiation successful
0810 11:30:15.781984 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16060,"thread_start_us":15911,"threads_started":1}
I20260810 11:30:15.784332 9823 tablet_service.cc:1467] Processing CreateTablet for tablet 08982a2248e14d23aa5c4cd25551c077 (DEFAULT_TABLE table=test-table [id=374f1ad4669f402c866e452d21ba3935]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:30:15.786547 9823 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 08982a2248e14d23aa5c4cd25551c077. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:30:15.786132 9681 tablet_service.cc:1467] Processing CreateTablet for tablet 08982a2248e14d23aa5c4cd25551c077 (DEFAULT_TABLE table=test-table [id=374f1ad4669f402c866e452d21ba3935]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:30:15.787858 9681 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 08982a2248e14d23aa5c4cd25551c077. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:30:15.788086 9540 tablet_service.cc:1467] Processing CreateTablet for tablet 08982a2248e14d23aa5c4cd25551c077 (DEFAULT_TABLE table=test-table [id=374f1ad4669f402c866e452d21ba3935]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:30:15.789655 9540 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 08982a2248e14d23aa5c4cd25551c077. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:30:15.801981 9913 tablet_bootstrap.cc:492] T 08982a2248e14d23aa5c4cd25551c077 P 90566adbea5742dea2a7249d96470451: Bootstrap starting.
I20260810 11:30:15.805658 9913 tablet_bootstrap.cc:654] T 08982a2248e14d23aa5c4cd25551c077 P 90566adbea5742dea2a7249d96470451: Neither blocks nor log segments found. Creating new log.
I20260810 11:30:15.806457 9914 tablet_bootstrap.cc:492] T 08982a2248e14d23aa5c4cd25551c077 P bb46349f489247b4973643a309634d45: Bootstrap starting.
I20260810 11:30:15.807279 9913 log.cc:824] T 08982a2248e14d23aa5c4cd25551c077 P 90566adbea5742dea2a7249d96470451: Log is configured to *not* fsync() on all Append() calls
I20260810 11:30:15.811528 9913 tablet_bootstrap.cc:492] T 08982a2248e14d23aa5c4cd25551c077 P 90566adbea5742dea2a7249d96470451: No bootstrap required, opened a new log
I20260810 11:30:15.811955 9913 ts_tablet_manager.cc:1397] T 08982a2248e14d23aa5c4cd25551c077 P 90566adbea5742dea2a7249d96470451: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.006s
I20260810 11:30:15.812000 9914 tablet_bootstrap.cc:654] T 08982a2248e14d23aa5c4cd25551c077 P bb46349f489247b4973643a309634d45: Neither blocks nor log segments found. Creating new log.
I20260810 11:30:15.813859 9915 tablet_bootstrap.cc:492] T 08982a2248e14d23aa5c4cd25551c077 P c88a7a11405f4a7699f9573f7dcfd4c0: Bootstrap starting.
I20260810 11:30:15.813989 9914 log.cc:824] T 08982a2248e14d23aa5c4cd25551c077 P bb46349f489247b4973643a309634d45: Log is configured to *not* fsync() on all Append() calls
I20260810 11:30:15.818384 9915 tablet_bootstrap.cc:654] T 08982a2248e14d23aa5c4cd25551c077 P c88a7a11405f4a7699f9573f7dcfd4c0: Neither blocks nor log segments found. Creating new log.
I20260810 11:30:15.818686 9914 tablet_bootstrap.cc:492] T 08982a2248e14d23aa5c4cd25551c077 P bb46349f489247b4973643a309634d45: No bootstrap required, opened a new log
I20260810 11:30:15.819118 9914 ts_tablet_manager.cc:1397] T 08982a2248e14d23aa5c4cd25551c077 P bb46349f489247b4973643a309634d45: Time spent bootstrapping tablet: real 0.013s user 0.010s sys 0.000s
I20260810 11:30:15.820143 9915 log.cc:824] T 08982a2248e14d23aa5c4cd25551c077 P c88a7a11405f4a7699f9573f7dcfd4c0: Log is configured to *not* fsync() on all Append() calls
I20260810 11:30:15.824793 9913 raft_consensus.cc:348] T 08982a2248e14d23aa5c4cd25551c077 P 90566adbea5742dea2a7249d96470451 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90566adbea5742dea2a7249d96470451" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 41353 } } peers { permanent_uuid: "bb46349f489247b4973643a309634d45" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 44751 } } peers { permanent_uuid: "c88a7a11405f4a7699f9573f7dcfd4c0" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 39295 } }
I20260810 11:30:15.825515 9913 raft_consensus.cc:374] T 08982a2248e14d23aa5c4cd25551c077 P 90566adbea5742dea2a7249d96470451 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:30:15.825681 9913 raft_consensus.cc:729] T 08982a2248e14d23aa5c4cd25551c077 P 90566adbea5742dea2a7249d96470451 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90566adbea5742dea2a7249d96470451, State: Initialized, Role: FOLLOWER
I20260810 11:30:15.826318 9915 tablet_bootstrap.cc:492] T 08982a2248e14d23aa5c4cd25551c077 P c88a7a11405f4a7699f9573f7dcfd4c0: No bootstrap required, opened a new log
I20260810 11:30:15.826645 9915 ts_tablet_manager.cc:1397] T 08982a2248e14d23aa5c4cd25551c077 P c88a7a11405f4a7699f9573f7dcfd4c0: Time spent bootstrapping tablet: real 0.018s user 0.012s sys 0.000s
I20260810 11:30:15.826474 9913 consensus_queue.cc:260] T 08982a2248e14d23aa5c4cd25551c077 P 90566adbea5742dea2a7249d96470451 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90566adbea5742dea2a7249d96470451" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 41353 } } peers { permanent_uuid: "bb46349f489247b4973643a309634d45" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 44751 } } peers { permanent_uuid: "c88a7a11405f4a7699f9573f7dcfd4c0" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 39295 } }
I20260810 11:30:15.827986 9894 heartbeater.cc:503] Master 127.8.18.254:35405 was elected leader, sending a full tablet report...
I20260810 11:30:15.828809 9913 ts_tablet_manager.cc:1428] T 08982a2248e14d23aa5c4cd25551c077 P 90566adbea5742dea2a7249d96470451: Time spent starting tablet: real 0.017s user 0.016s sys 0.002s
I20260810 11:30:15.831338 9914 raft_consensus.cc:348] T 08982a2248e14d23aa5c4cd25551c077 P bb46349f489247b4973643a309634d45 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90566adbea5742dea2a7249d96470451" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 41353 } } peers { permanent_uuid: "bb46349f489247b4973643a309634d45" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 44751 } } peers { permanent_uuid: "c88a7a11405f4a7699f9573f7dcfd4c0" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 39295 } }
I20260810 11:30:15.831928 9914 raft_consensus.cc:374] T 08982a2248e14d23aa5c4cd25551c077 P bb46349f489247b4973643a309634d45 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:30:15.832062 9914 raft_consensus.cc:729] T 08982a2248e14d23aa5c4cd25551c077 P bb46349f489247b4973643a309634d45 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb46349f489247b4973643a309634d45, State: Initialized, Role: FOLLOWER
I20260810 11:30:15.832924 9914 consensus_queue.cc:260] T 08982a2248e14d23aa5c4cd25551c077 P bb46349f489247b4973643a309634d45 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90566adbea5742dea2a7249d96470451" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 41353 } } peers { permanent_uuid: "bb46349f489247b4973643a309634d45" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 44751 } } peers { permanent_uuid: "c88a7a11405f4a7699f9573f7dcfd4c0" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 39295 } }
I20260810 11:30:15.834296 9752 heartbeater.cc:503] Master 127.8.18.254:35405 was elected leader, sending a full tablet report...
I20260810 11:30:15.835232 9914 ts_tablet_manager.cc:1428] T 08982a2248e14d23aa5c4cd25551c077 P bb46349f489247b4973643a309634d45: Time spent starting tablet: real 0.016s user 0.010s sys 0.008s
W20260810 11:30:15.837280 9753 tablet.cc:2369] T 08982a2248e14d23aa5c4cd25551c077 P bb46349f489247b4973643a309634d45: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 11:30:15.838445 9915 raft_consensus.cc:348] T 08982a2248e14d23aa5c4cd25551c077 P c88a7a11405f4a7699f9573f7dcfd4c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90566adbea5742dea2a7249d96470451" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 41353 } } peers { permanent_uuid: "bb46349f489247b4973643a309634d45" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 44751 } } peers { permanent_uuid: "c88a7a11405f4a7699f9573f7dcfd4c0" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 39295 } }
I20260810 11:30:15.839020 9915 raft_consensus.cc:374] T 08982a2248e14d23aa5c4cd25551c077 P c88a7a11405f4a7699f9573f7dcfd4c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:30:15.839135 9915 raft_consensus.cc:729] T 08982a2248e14d23aa5c4cd25551c077 P c88a7a11405f4a7699f9573f7dcfd4c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c88a7a11405f4a7699f9573f7dcfd4c0, State: Initialized, Role: FOLLOWER
I20260810 11:30:15.839885 9915 consensus_queue.cc:260] T 08982a2248e14d23aa5c4cd25551c077 P c88a7a11405f4a7699f9573f7dcfd4c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90566adbea5742dea2a7249d96470451" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 41353 } } peers { permanent_uuid: "bb46349f489247b4973643a309634d45" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 44751 } } peers { permanent_uuid: "c88a7a11405f4a7699f9573f7dcfd4c0" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 39295 } }
I20260810 11:30:15.842154 9915 ts_tablet_manager.cc:1428] T 08982a2248e14d23aa5c4cd25551c077 P c88a7a11405f4a7699f9573f7dcfd4c0: Time spent starting tablet: real 0.015s user 0.010s sys 0.005s
W20260810 11:30:15.908437 9895 tablet.cc:2369] T 08982a2248e14d23aa5c4cd25551c077 P 90566adbea5742dea2a7249d96470451: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 11:30:16.013968 9612 tablet.cc:2369] T 08982a2248e14d23aa5c4cd25551c077 P c88a7a11405f4a7699f9573f7dcfd4c0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 11:30:16.062852 9920 raft_consensus.cc:482] T 08982a2248e14d23aa5c4cd25551c077 P bb46349f489247b4973643a309634d45 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 11:30:16.063309 9920 raft_consensus.cc:504] T 08982a2248e14d23aa5c4cd25551c077 P bb46349f489247b4973643a309634d45 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90566adbea5742dea2a7249d96470451" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 41353 } } peers { permanent_uuid: "bb46349f489247b4973643a309634d45" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 44751 } } peers { permanent_uuid: "c88a7a11405f4a7699f9573f7dcfd4c0" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 39295 } }
I20260810 11:30:16.065241 9920 leader_election.cc:288] T 08982a2248e14d23aa5c4cd25551c077 P bb46349f489247b4973643a309634d45 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 90566adbea5742dea2a7249d96470451 (127.8.18.195:41353), c88a7a11405f4a7699f9573f7dcfd4c0 (127.8.18.193:39295)
I20260810 11:30:16.076002 9912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:16.065898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.194:41315 (local address 127.8.18.193:39295)
0810 11:30:16.066124 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0810 11:30:16.066130 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:30:16.067330 (+ 1200us) server_negotiation.cc:407] Connection header received
0810 11:30:16.067596 (+ 266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:30:16.067600 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:30:16.067682 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:30:16.067789 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:30:16.068729 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:16.069852 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:16.072924 (+ 3072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:16.074438 (+ 1514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:16.074712 (+ 274us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:16.074801 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0810 11:30:16.075541 (+ 740us) server_negotiation.cc:299] Negotiation successful
0810 11:30:16.075714 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260810 11:30:16.076036 9908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:16.065706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.194:57451 (local address 127.8.18.195:41353)
0810 11:30:16.065982 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0810 11:30:16.065991 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 11:30:16.067265 (+ 1274us) server_negotiation.cc:407] Connection header received
0810 11:30:16.067573 (+ 308us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:30:16.067579 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:30:16.067665 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:30:16.067767 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:30:16.069121 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:16.070278 (+ 1157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:16.072956 (+ 2678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:16.074510 (+ 1554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:16.074711 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:16.074800 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0810 11:30:16.075541 (+ 741us) server_negotiation.cc:299] Negotiation successful
0810 11:30:16.075691 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260810 11:30:16.076685 9922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:16.065809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.195:41353 (local address 127.8.18.194:57451)
0810 11:30:16.067117 (+ 1308us) negotiation.cc:107] Waiting for socket to connect
0810 11:30:16.067137 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 11:30:16.067411 (+ 274us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:30:16.068175 (+ 764us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:30:16.068179 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:30:16.068202 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:30:16.068951 (+ 749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:30:16.068962 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:30:16.070439 (+ 1477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:30:16.070451 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:30:16.072783 (+ 2332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:30:16.072797 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:30:16.074940 (+ 2143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:30:16.074952 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:30:16.075267 (+ 315us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:16.075364 (+ 97us) client_negotiation.cc:769] Sending connection context
0810 11:30:16.075479 (+ 115us) client_negotiation.cc:240] Negotiation successful
0810 11:30:16.075690 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1102,"mutex_wait_us":119,"spinlock_wait_cycles":1920,"thread_start_us":653,"threads_started":1}
I20260810 11:30:16.076794 9923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:16.066186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.193:39295 (local address 127.8.18.194:41315)
0810 11:30:16.067198 (+ 1012us) negotiation.cc:107] Waiting for socket to connect
0810 11:30:16.067219 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 11:30:16.067429 (+ 210us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:30:16.067964 (+ 535us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:30:16.067968 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:30:16.068004 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:30:16.068508 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:30:16.068524 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:30:16.070009 (+ 1485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:30:16.070014 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:30:16.072687 (+ 2673us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:30:16.072714 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:30:16.074948 (+ 2234us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:30:16.074953 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:30:16.075266 (+ 313us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:16.075359 (+ 93us) client_negotiation.cc:769] Sending connection context
0810 11:30:16.075479 (+ 120us) client_negotiation.cc:240] Negotiation successful
0810 11:30:16.075690 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":823,"mutex_wait_us":84,"spinlock_wait_cycles":60160,"thread_start_us":515,"threads_started":1}
I20260810 11:30:16.077991 9843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08982a2248e14d23aa5c4cd25551c077" candidate_uuid: "bb46349f489247b4973643a309634d45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90566adbea5742dea2a7249d96470451" is_pre_election: true
I20260810 11:30:16.077960 9562 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08982a2248e14d23aa5c4cd25551c077" candidate_uuid: "bb46349f489247b4973643a309634d45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c88a7a11405f4a7699f9573f7dcfd4c0" is_pre_election: true
I20260810 11:30:16.078560 9843 raft_consensus.cc:2454] T 08982a2248e14d23aa5c4cd25551c077 P 90566adbea5742dea2a7249d96470451 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bb46349f489247b4973643a309634d45 in term 0.
I20260810 11:30:16.078554 9562 raft_consensus.cc:2454] T 08982a2248e14d23aa5c4cd25551c077 P c88a7a11405f4a7699f9573f7dcfd4c0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bb46349f489247b4973643a309634d45 in term 0.
I20260810 11:30:16.079348 9633 leader_election.cc:302] T 08982a2248e14d23aa5c4cd25551c077 P bb46349f489247b4973643a309634d45 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90566adbea5742dea2a7249d96470451, bb46349f489247b4973643a309634d45; no voters:
I20260810 11:30:16.079881 9920 raft_consensus.cc:2781] T 08982a2248e14d23aa5c4cd25551c077 P bb46349f489247b4973643a309634d45 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 11:30:16.080101 9920 raft_consensus.cc:482] T 08982a2248e14d23aa5c4cd25551c077 P bb46349f489247b4973643a309634d45 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 11:30:16.080220 9920 raft_consensus.cc:3037] T 08982a2248e14d23aa5c4cd25551c077 P bb46349f489247b4973643a309634d45 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:30:16.083523 9920 raft_consensus.cc:504] T 08982a2248e14d23aa5c4cd25551c077 P bb46349f489247b4973643a309634d45 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90566adbea5742dea2a7249d96470451" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 41353 } } peers { permanent_uuid: "bb46349f489247b4973643a309634d45" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 44751 } } peers { permanent_uuid: "c88a7a11405f4a7699f9573f7dcfd4c0" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 39295 } }
I20260810 11:30:16.084676 9920 leader_election.cc:288] T 08982a2248e14d23aa5c4cd25551c077 P bb46349f489247b4973643a309634d45 [CANDIDATE]: Term 1 election: Requested vote from peers 90566adbea5742dea2a7249d96470451 (127.8.18.195:41353), c88a7a11405f4a7699f9573f7dcfd4c0 (127.8.18.193:39295)
I20260810 11:30:16.085482 9843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08982a2248e14d23aa5c4cd25551c077" candidate_uuid: "bb46349f489247b4973643a309634d45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90566adbea5742dea2a7249d96470451"
I20260810 11:30:16.085572 9562 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08982a2248e14d23aa5c4cd25551c077" candidate_uuid: "bb46349f489247b4973643a309634d45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c88a7a11405f4a7699f9573f7dcfd4c0"
I20260810 11:30:16.085824 9843 raft_consensus.cc:3037] T 08982a2248e14d23aa5c4cd25551c077 P 90566adbea5742dea2a7249d96470451 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:30:16.085897 9562 raft_consensus.cc:3037] T 08982a2248e14d23aa5c4cd25551c077 P c88a7a11405f4a7699f9573f7dcfd4c0 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:30:16.089381 9843 raft_consensus.cc:2454] T 08982a2248e14d23aa5c4cd25551c077 P 90566adbea5742dea2a7249d96470451 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bb46349f489247b4973643a309634d45 in term 1.
I20260810 11:30:16.090150 9633 leader_election.cc:302] T 08982a2248e14d23aa5c4cd25551c077 P bb46349f489247b4973643a309634d45 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90566adbea5742dea2a7249d96470451, bb46349f489247b4973643a309634d45; no voters:
I20260810 11:30:16.090214 9562 raft_consensus.cc:2454] T 08982a2248e14d23aa5c4cd25551c077 P c88a7a11405f4a7699f9573f7dcfd4c0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bb46349f489247b4973643a309634d45 in term 1.
I20260810 11:30:16.090590 9920 raft_consensus.cc:2781] T 08982a2248e14d23aa5c4cd25551c077 P bb46349f489247b4973643a309634d45 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:30:16.091305 9920 raft_consensus.cc:686] T 08982a2248e14d23aa5c4cd25551c077 P bb46349f489247b4973643a309634d45 [term 1 LEADER]: Becoming Leader. State: Replica: bb46349f489247b4973643a309634d45, State: Running, Role: LEADER
I20260810 11:30:16.091907 9920 consensus_queue.cc:237] T 08982a2248e14d23aa5c4cd25551c077 P bb46349f489247b4973643a309634d45 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90566adbea5742dea2a7249d96470451" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 41353 } } peers { permanent_uuid: "bb46349f489247b4973643a309634d45" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 44751 } } peers { permanent_uuid: "c88a7a11405f4a7699f9573f7dcfd4c0" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 39295 } }
I20260810 11:30:16.097872 9401 catalog_manager.cc:5495] T 08982a2248e14d23aa5c4cd25551c077 P bb46349f489247b4973643a309634d45 reported cstate change: term changed from 0 to 1, leader changed from <none> to bb46349f489247b4973643a309634d45 (127.8.18.194). New cstate: current_term: 1 leader_uuid: "bb46349f489247b4973643a309634d45" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90566adbea5742dea2a7249d96470451" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 41353 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bb46349f489247b4973643a309634d45" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 44751 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c88a7a11405f4a7699f9573f7dcfd4c0" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 39295 } health_report { overall_health: UNKNOWN } } }
I20260810 11:30:16.169767 9911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:16.163628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35780 (local address 127.8.18.194:44751)
0810 11:30:16.163962 (+ 334us) server_negotiation.cc:206] Beginning negotiation
0810 11:30:16.163969 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:30:16.164032 (+ 63us) server_negotiation.cc:407] Connection header received
0810 11:30:16.164344 (+ 312us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:30:16.164348 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:30:16.164446 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:30:16.164588 (+ 142us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 11:30:16.164943 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:16.165853 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:16.167241 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:16.167680 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:16.167789 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:16.168352 (+ 563us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 11:30:16.169040 (+ 688us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 11:30:16.169160 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0810 11:30:16.169305 (+ 145us) server_negotiation.cc:299] Negotiation successful
0810 11:30:16.169380 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147}
I20260810 11:30:16.187182 9843 raft_consensus.cc:1261] T 08982a2248e14d23aa5c4cd25551c077 P 90566adbea5742dea2a7249d96470451 [term 1 FOLLOWER]: Refusing update from remote peer bb46349f489247b4973643a309634d45: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 11:30:16.188687 9562 raft_consensus.cc:1261] T 08982a2248e14d23aa5c4cd25551c077 P c88a7a11405f4a7699f9573f7dcfd4c0 [term 1 FOLLOWER]: Refusing update from remote peer bb46349f489247b4973643a309634d45: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 11:30:16.189224 9924 consensus_queue.cc:1035] T 08982a2248e14d23aa5c4cd25551c077 P bb46349f489247b4973643a309634d45 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90566adbea5742dea2a7249d96470451" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 41353 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 11:30:16.190023 9920 consensus_queue.cc:1035] T 08982a2248e14d23aa5c4cd25551c077 P bb46349f489247b4973643a309634d45 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c88a7a11405f4a7699f9573f7dcfd4c0" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 39295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 11:30:16.205267 9927 mvcc.cc:205] Tried to move back new op lower bound from 7316936360663715840 to 7316936360317161472. Current Snapshot: MvccSnapshot[applied={T|T < 7316936360663715840}]
I20260810 11:30:16.237145 9401 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48576:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 11:30:16.237634 9401 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48576:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 11:30:16.242516 9401 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 41a39e4b02e143cda20aa5740d974b27: Sending DeleteTablet for 3 replicas of tablet 08982a2248e14d23aa5c4cd25551c077
I20260810 11:30:16.244383 9823 tablet_service.cc:1514] Processing DeleteTablet for tablet 08982a2248e14d23aa5c4cd25551c077 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:30:16 UTC) from {username='kudu', principal='kudu/127.8.18.254@KRBTEST.COM'} at 127.0.0.1:43176
I20260810 11:30:16.244496 9540 tablet_service.cc:1514] Processing DeleteTablet for tablet 08982a2248e14d23aa5c4cd25551c077 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:30:16 UTC) from {username='kudu', principal='kudu/127.8.18.254@KRBTEST.COM'} at 127.0.0.1:49340
I20260810 11:30:16.244422 9681 tablet_service.cc:1514] Processing DeleteTablet for tablet 08982a2248e14d23aa5c4cd25551c077 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:30:16 UTC) from {username='kudu', principal='kudu/127.8.18.254@KRBTEST.COM'} at 127.0.0.1:35768
I20260810 11:30:16.251508 9401 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48576:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 11:30:16.254240 9401 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 11:30:16.256845 9939 tablet_replica.cc:330] T 08982a2248e14d23aa5c4cd25551c077 P c88a7a11405f4a7699f9573f7dcfd4c0: stopping tablet replica
I20260810 11:30:16.257751 9939 raft_consensus.cc:2229] T 08982a2248e14d23aa5c4cd25551c077 P c88a7a11405f4a7699f9573f7dcfd4c0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 11:30:16.258528 9939 raft_consensus.cc:2258] T 08982a2248e14d23aa5c4cd25551c077 P c88a7a11405f4a7699f9573f7dcfd4c0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:30:16.259724 9938 tablet_replica.cc:330] T 08982a2248e14d23aa5c4cd25551c077 P 90566adbea5742dea2a7249d96470451: stopping tablet replica
I20260810 11:30:16.260303 9938 raft_consensus.cc:2229] T 08982a2248e14d23aa5c4cd25551c077 P 90566adbea5742dea2a7249d96470451 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 11:30:16.260928 9938 raft_consensus.cc:2258] T 08982a2248e14d23aa5c4cd25551c077 P 90566adbea5742dea2a7249d96470451 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:30:16.263808 9940 tablet_replica.cc:330] T 08982a2248e14d23aa5c4cd25551c077 P bb46349f489247b4973643a309634d45: stopping tablet replica
I20260810 11:30:16.263911 9939 ts_tablet_manager.cc:1905] T 08982a2248e14d23aa5c4cd25551c077 P c88a7a11405f4a7699f9573f7dcfd4c0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:30:16.264385 9940 raft_consensus.cc:2229] T 08982a2248e14d23aa5c4cd25551c077 P bb46349f489247b4973643a309634d45 [term 1 LEADER]: Raft consensus shutting down.
I20260810 11:30:16.265440 9940 raft_consensus.cc:2258] T 08982a2248e14d23aa5c4cd25551c077 P bb46349f489247b4973643a309634d45 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:30:16.268858 9823 tablet_service.cc:1467] Processing CreateTablet for tablet 4042b507d68741398ce131a4ebc5bba7 (DEFAULT_TABLE table=test-table [id=722cc5a49f6343ea9a77c7f5911f0065]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:30:16.268898 9540 tablet_service.cc:1467] Processing CreateTablet for tablet 4042b507d68741398ce131a4ebc5bba7 (DEFAULT_TABLE table=test-table [id=722cc5a49f6343ea9a77c7f5911f0065]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:30:16.269461 9823 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4042b507d68741398ce131a4ebc5bba7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:30:16.269480 9540 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4042b507d68741398ce131a4ebc5bba7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:30:16.269536 9681 tablet_service.cc:1467] Processing CreateTablet for tablet 4042b507d68741398ce131a4ebc5bba7 (DEFAULT_TABLE table=test-table [id=722cc5a49f6343ea9a77c7f5911f0065]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:30:16.270073 9681 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4042b507d68741398ce131a4ebc5bba7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:30:16.277621 9913 tablet_bootstrap.cc:492] T 4042b507d68741398ce131a4ebc5bba7 P 90566adbea5742dea2a7249d96470451: Bootstrap starting.
I20260810 11:30:16.278790 9915 tablet_bootstrap.cc:492] T 4042b507d68741398ce131a4ebc5bba7 P c88a7a11405f4a7699f9573f7dcfd4c0: Bootstrap starting.
I20260810 11:30:16.279549 9940 ts_tablet_manager.cc:1905] T 08982a2248e14d23aa5c4cd25551c077 P bb46349f489247b4973643a309634d45: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:30:16.279953 9913 tablet_bootstrap.cc:654] T 4042b507d68741398ce131a4ebc5bba7 P 90566adbea5742dea2a7249d96470451: Neither blocks nor log segments found. Creating new log.
I20260810 11:30:16.280897 9915 tablet_bootstrap.cc:654] T 4042b507d68741398ce131a4ebc5bba7 P c88a7a11405f4a7699f9573f7dcfd4c0: Neither blocks nor log segments found. Creating new log.
I20260810 11:30:16.290359 9915 tablet_bootstrap.cc:492] T 4042b507d68741398ce131a4ebc5bba7 P c88a7a11405f4a7699f9573f7dcfd4c0: No bootstrap required, opened a new log
I20260810 11:30:16.290582 9915 ts_tablet_manager.cc:1397] T 4042b507d68741398ce131a4ebc5bba7 P c88a7a11405f4a7699f9573f7dcfd4c0: Time spent bootstrapping tablet: real 0.012s user 0.003s sys 0.003s
I20260810 11:30:16.291759 9915 raft_consensus.cc:348] T 4042b507d68741398ce131a4ebc5bba7 P c88a7a11405f4a7699f9573f7dcfd4c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c88a7a11405f4a7699f9573f7dcfd4c0" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 39295 } } peers { permanent_uuid: "bb46349f489247b4973643a309634d45" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 44751 } } peers { permanent_uuid: "90566adbea5742dea2a7249d96470451" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 41353 } }
I20260810 11:30:16.292098 9915 raft_consensus.cc:374] T 4042b507d68741398ce131a4ebc5bba7 P c88a7a11405f4a7699f9573f7dcfd4c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:30:16.292218 9915 raft_consensus.cc:729] T 4042b507d68741398ce131a4ebc5bba7 P c88a7a11405f4a7699f9573f7dcfd4c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c88a7a11405f4a7699f9573f7dcfd4c0, State: Initialized, Role: FOLLOWER
I20260810 11:30:16.292650 9915 consensus_queue.cc:260] T 4042b507d68741398ce131a4ebc5bba7 P c88a7a11405f4a7699f9573f7dcfd4c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c88a7a11405f4a7699f9573f7dcfd4c0" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 39295 } } peers { permanent_uuid: "bb46349f489247b4973643a309634d45" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 44751 } } peers { permanent_uuid: "90566adbea5742dea2a7249d96470451" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 41353 } }
I20260810 11:30:16.294183 9938 ts_tablet_manager.cc:1905] T 08982a2248e14d23aa5c4cd25551c077 P 90566adbea5742dea2a7249d96470451: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:30:16.296098 9939 ts_tablet_manager.cc:1918] T 08982a2248e14d23aa5c4cd25551c077 P c88a7a11405f4a7699f9573f7dcfd4c0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 11:30:16.296409 9939 log.cc:1196] T 08982a2248e14d23aa5c4cd25551c077 P c88a7a11405f4a7699f9573f7dcfd4c0: Deleting WAL directory at /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786361401922362-8267-0/minicluster-data/ts-0/wal/wals/08982a2248e14d23aa5c4cd25551c077
I20260810 11:30:16.296531 9915 ts_tablet_manager.cc:1428] T 4042b507d68741398ce131a4ebc5bba7 P c88a7a11405f4a7699f9573f7dcfd4c0: Time spent starting tablet: real 0.006s user 0.003s sys 0.000s
I20260810 11:30:16.297639 9914 tablet_bootstrap.cc:492] T 4042b507d68741398ce131a4ebc5bba7 P bb46349f489247b4973643a309634d45: Bootstrap starting.
I20260810 11:30:16.299216 9939 ts_tablet_manager.cc:1939] T 08982a2248e14d23aa5c4cd25551c077 P c88a7a11405f4a7699f9573f7dcfd4c0: Deleting consensus metadata
I20260810 11:30:16.300000 9914 tablet_bootstrap.cc:654] T 4042b507d68741398ce131a4ebc5bba7 P bb46349f489247b4973643a309634d45: Neither blocks nor log segments found. Creating new log.
I20260810 11:30:16.301728 9385 catalog_manager.cc:4841] TS c88a7a11405f4a7699f9573f7dcfd4c0 (127.8.18.193:39295): tablet 08982a2248e14d23aa5c4cd25551c077 (table test-table [id=374f1ad4669f402c866e452d21ba3935]) successfully deleted
I20260810 11:30:16.302073 9940 ts_tablet_manager.cc:1918] T 08982a2248e14d23aa5c4cd25551c077 P bb46349f489247b4973643a309634d45: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 11:30:16.302347 9940 log.cc:1196] T 08982a2248e14d23aa5c4cd25551c077 P bb46349f489247b4973643a309634d45: Deleting WAL directory at /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786361401922362-8267-0/minicluster-data/ts-1/wal/wals/08982a2248e14d23aa5c4cd25551c077
I20260810 11:30:16.303103 9940 ts_tablet_manager.cc:1939] T 08982a2248e14d23aa5c4cd25551c077 P bb46349f489247b4973643a309634d45: Deleting consensus metadata
I20260810 11:30:16.303942 9913 tablet_bootstrap.cc:492] T 4042b507d68741398ce131a4ebc5bba7 P 90566adbea5742dea2a7249d96470451: No bootstrap required, opened a new log
I20260810 11:30:16.304181 9913 ts_tablet_manager.cc:1397] T 4042b507d68741398ce131a4ebc5bba7 P 90566adbea5742dea2a7249d96470451: Time spent bootstrapping tablet: real 0.027s user 0.011s sys 0.001s
I20260810 11:30:16.305333 9388 catalog_manager.cc:4841] TS bb46349f489247b4973643a309634d45 (127.8.18.194:44751): tablet 08982a2248e14d23aa5c4cd25551c077 (table test-table [id=374f1ad4669f402c866e452d21ba3935]) successfully deleted
I20260810 11:30:16.305382 9913 raft_consensus.cc:348] T 4042b507d68741398ce131a4ebc5bba7 P 90566adbea5742dea2a7249d96470451 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c88a7a11405f4a7699f9573f7dcfd4c0" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 39295 } } peers { permanent_uuid: "bb46349f489247b4973643a309634d45" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 44751 } } peers { permanent_uuid: "90566adbea5742dea2a7249d96470451" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 41353 } }
I20260810 11:30:16.305696 9913 raft_consensus.cc:374] T 4042b507d68741398ce131a4ebc5bba7 P 90566adbea5742dea2a7249d96470451 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:30:16.305925 9913 raft_consensus.cc:729] T 4042b507d68741398ce131a4ebc5bba7 P 90566adbea5742dea2a7249d96470451 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90566adbea5742dea2a7249d96470451, State: Initialized, Role: FOLLOWER
I20260810 11:30:16.306442 9913 consensus_queue.cc:260] T 4042b507d68741398ce131a4ebc5bba7 P 90566adbea5742dea2a7249d96470451 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c88a7a11405f4a7699f9573f7dcfd4c0" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 39295 } } peers { permanent_uuid: "bb46349f489247b4973643a309634d45" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 44751 } } peers { permanent_uuid: "90566adbea5742dea2a7249d96470451" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 41353 } }
I20260810 11:30:16.310395 9913 ts_tablet_manager.cc:1428] T 4042b507d68741398ce131a4ebc5bba7 P 90566adbea5742dea2a7249d96470451: Time spent starting tablet: real 0.006s user 0.000s sys 0.003s
I20260810 11:30:16.311712 9938 ts_tablet_manager.cc:1918] T 08982a2248e14d23aa5c4cd25551c077 P 90566adbea5742dea2a7249d96470451: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 11:30:16.312052 9938 log.cc:1196] T 08982a2248e14d23aa5c4cd25551c077 P 90566adbea5742dea2a7249d96470451: Deleting WAL directory at /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786361401922362-8267-0/minicluster-data/ts-2/wal/wals/08982a2248e14d23aa5c4cd25551c077
I20260810 11:30:16.313007 9938 ts_tablet_manager.cc:1939] T 08982a2248e14d23aa5c4cd25551c077 P 90566adbea5742dea2a7249d96470451: Deleting consensus metadata
I20260810 11:30:16.315932 9387 catalog_manager.cc:4841] TS 90566adbea5742dea2a7249d96470451 (127.8.18.195:41353): tablet 08982a2248e14d23aa5c4cd25551c077 (table test-table [id=374f1ad4669f402c866e452d21ba3935]) successfully deleted
I20260810 11:30:16.317550 9914 tablet_bootstrap.cc:492] T 4042b507d68741398ce131a4ebc5bba7 P bb46349f489247b4973643a309634d45: No bootstrap required, opened a new log
I20260810 11:30:16.317754 9914 ts_tablet_manager.cc:1397] T 4042b507d68741398ce131a4ebc5bba7 P bb46349f489247b4973643a309634d45: Time spent bootstrapping tablet: real 0.020s user 0.008s sys 0.009s
I20260810 11:30:16.318651 9914 raft_consensus.cc:348] T 4042b507d68741398ce131a4ebc5bba7 P bb46349f489247b4973643a309634d45 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c88a7a11405f4a7699f9573f7dcfd4c0" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 39295 } } peers { permanent_uuid: "bb46349f489247b4973643a309634d45" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 44751 } } peers { permanent_uuid: "90566adbea5742dea2a7249d96470451" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 41353 } }
I20260810 11:30:16.318848 9914 raft_consensus.cc:374] T 4042b507d68741398ce131a4ebc5bba7 P bb46349f489247b4973643a309634d45 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:30:16.318970 9914 raft_consensus.cc:729] T 4042b507d68741398ce131a4ebc5bba7 P bb46349f489247b4973643a309634d45 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb46349f489247b4973643a309634d45, State: Initialized, Role: FOLLOWER
I20260810 11:30:16.319267 9914 consensus_queue.cc:260] T 4042b507d68741398ce131a4ebc5bba7 P bb46349f489247b4973643a309634d45 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c88a7a11405f4a7699f9573f7dcfd4c0" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 39295 } } peers { permanent_uuid: "bb46349f489247b4973643a309634d45" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 44751 } } peers { permanent_uuid: "90566adbea5742dea2a7249d96470451" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 41353 } }
I20260810 11:30:16.319974 9914 ts_tablet_manager.cc:1428] T 4042b507d68741398ce131a4ebc5bba7 P bb46349f489247b4973643a309634d45: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260810 11:30:16.421207 9921 raft_consensus.cc:482] T 4042b507d68741398ce131a4ebc5bba7 P c88a7a11405f4a7699f9573f7dcfd4c0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 11:30:16.421581 9921 raft_consensus.cc:504] T 4042b507d68741398ce131a4ebc5bba7 P c88a7a11405f4a7699f9573f7dcfd4c0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c88a7a11405f4a7699f9573f7dcfd4c0" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 39295 } } peers { permanent_uuid: "bb46349f489247b4973643a309634d45" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 44751 } } peers { permanent_uuid: "90566adbea5742dea2a7249d96470451" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 41353 } }
I20260810 11:30:16.424182 9921 leader_election.cc:288] T 4042b507d68741398ce131a4ebc5bba7 P c88a7a11405f4a7699f9573f7dcfd4c0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bb46349f489247b4973643a309634d45 (127.8.18.194:44751), 90566adbea5742dea2a7249d96470451 (127.8.18.195:41353)
I20260810 11:30:16.434607 9924 raft_consensus.cc:482] T 4042b507d68741398ce131a4ebc5bba7 P bb46349f489247b4973643a309634d45 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 11:30:16.434872 9924 raft_consensus.cc:504] T 4042b507d68741398ce131a4ebc5bba7 P bb46349f489247b4973643a309634d45 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c88a7a11405f4a7699f9573f7dcfd4c0" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 39295 } } peers { permanent_uuid: "bb46349f489247b4973643a309634d45" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 44751 } } peers { permanent_uuid: "90566adbea5742dea2a7249d96470451" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 41353 } }
I20260810 11:30:16.435729 9924 leader_election.cc:288] T 4042b507d68741398ce131a4ebc5bba7 P bb46349f489247b4973643a309634d45 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c88a7a11405f4a7699f9573f7dcfd4c0 (127.8.18.193:39295), 90566adbea5742dea2a7249d96470451 (127.8.18.195:41353)
I20260810 11:30:16.436264 9945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:16.425737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.195:41353 (local address 127.8.18.193:49207)
0810 11:30:16.426431 (+ 694us) negotiation.cc:107] Waiting for socket to connect
0810 11:30:16.426460 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0810 11:30:16.426590 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:30:16.427156 (+ 566us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:30:16.427163 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:30:16.427202 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:30:16.427726 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:30:16.427744 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:30:16.429735 (+ 1991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:30:16.429743 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:30:16.432692 (+ 2949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:30:16.432713 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:30:16.435185 (+ 2472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:30:16.435190 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:30:16.435400 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:16.435437 (+ 37us) client_negotiation.cc:769] Sending connection context
0810 11:30:16.435618 (+ 181us) client_negotiation.cc:240] Negotiation successful
0810 11:30:16.435789 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":545,"thread_start_us":1726,"threads_started":1}
I20260810 11:30:16.436620 9843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4042b507d68741398ce131a4ebc5bba7" candidate_uuid: "bb46349f489247b4973643a309634d45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90566adbea5742dea2a7249d96470451" is_pre_election: true
I20260810 11:30:16.436281 9562 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4042b507d68741398ce131a4ebc5bba7" candidate_uuid: "bb46349f489247b4973643a309634d45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c88a7a11405f4a7699f9573f7dcfd4c0" is_pre_election: true
I20260810 11:30:16.436947 9562 raft_consensus.cc:2454] T 4042b507d68741398ce131a4ebc5bba7 P c88a7a11405f4a7699f9573f7dcfd4c0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bb46349f489247b4973643a309634d45 in term 0.
I20260810 11:30:16.436947 9843 raft_consensus.cc:2454] T 4042b507d68741398ce131a4ebc5bba7 P 90566adbea5742dea2a7249d96470451 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bb46349f489247b4973643a309634d45 in term 0.
I20260810 11:30:16.437536 9633 leader_election.cc:302] T 4042b507d68741398ce131a4ebc5bba7 P bb46349f489247b4973643a309634d45 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90566adbea5742dea2a7249d96470451, bb46349f489247b4973643a309634d45; no voters:
I20260810 11:30:16.437825 9908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:16.426651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.193:49207 (local address 127.8.18.195:41353)
0810 11:30:16.426832 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0810 11:30:16.426838 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:30:16.426856 (+ 18us) server_negotiation.cc:407] Connection header received
0810 11:30:16.426925 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:30:16.426930 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:30:16.427018 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:30:16.427117 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:30:16.427985 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:16.429462 (+ 1477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:16.432944 (+ 3482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:16.434647 (+ 1703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:16.434836 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:16.434912 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0810 11:30:16.436371 (+ 1459us) server_negotiation.cc:299] Negotiation successful
0810 11:30:16.436527 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260810 11:30:16.437953 9924 raft_consensus.cc:2781] T 4042b507d68741398ce131a4ebc5bba7 P bb46349f489247b4973643a309634d45 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 11:30:16.438117 9924 raft_consensus.cc:482] T 4042b507d68741398ce131a4ebc5bba7 P bb46349f489247b4973643a309634d45 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 11:30:16.438385 9924 raft_consensus.cc:3037] T 4042b507d68741398ce131a4ebc5bba7 P bb46349f489247b4973643a309634d45 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:30:16.438516 9911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:16.425057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.193:54497 (local address 127.8.18.194:44751)
0810 11:30:16.425257 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0810 11:30:16.425263 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:30:16.426104 (+ 841us) server_negotiation.cc:407] Connection header received
0810 11:30:16.426195 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:30:16.426201 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:30:16.426280 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:30:16.426390 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:30:16.428308 (+ 1918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:16.429579 (+ 1271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:16.432685 (+ 3106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:16.434148 (+ 1463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:16.434296 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:16.434387 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0810 11:30:16.437486 (+ 3099us) server_negotiation.cc:299] Negotiation successful
0810 11:30:16.437790 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260810 11:30:16.439499 9944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:16.424577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.194:44751 (local address 127.8.18.193:54497)
0810 11:30:16.425401 (+ 824us) negotiation.cc:107] Waiting for socket to connect
0810 11:30:16.425456 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0810 11:30:16.425668 (+ 212us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:30:16.427668 (+ 2000us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:30:16.427677 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:30:16.427709 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:30:16.428090 (+ 381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:30:16.428102 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:30:16.429800 (+ 1698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:30:16.429805 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:30:16.432392 (+ 2587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:30:16.432413 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:30:16.436946 (+ 4533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:30:16.436954 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:30:16.437237 (+ 283us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:16.437275 (+ 38us) client_negotiation.cc:769] Sending connection context
0810 11:30:16.438903 (+ 1628us) client_negotiation.cc:240] Negotiation successful
0810 11:30:16.439081 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":642,"mutex_wait_us":41,"thread_start_us":2709,"threads_started":1}
I20260810 11:30:16.440753 9702 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4042b507d68741398ce131a4ebc5bba7" candidate_uuid: "c88a7a11405f4a7699f9573f7dcfd4c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb46349f489247b4973643a309634d45" is_pre_election: true
I20260810 11:30:16.441430 9843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4042b507d68741398ce131a4ebc5bba7" candidate_uuid: "c88a7a11405f4a7699f9573f7dcfd4c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90566adbea5742dea2a7249d96470451" is_pre_election: true
I20260810 11:30:16.441747 9843 raft_consensus.cc:2454] T 4042b507d68741398ce131a4ebc5bba7 P 90566adbea5742dea2a7249d96470451 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c88a7a11405f4a7699f9573f7dcfd4c0 in term 0.
I20260810 11:30:16.441677 9924 raft_consensus.cc:504] T 4042b507d68741398ce131a4ebc5bba7 P bb46349f489247b4973643a309634d45 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c88a7a11405f4a7699f9573f7dcfd4c0" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 39295 } } peers { permanent_uuid: "bb46349f489247b4973643a309634d45" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 44751 } } peers { permanent_uuid: "90566adbea5742dea2a7249d96470451" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 41353 } }
I20260810 11:30:16.442348 9702 raft_consensus.cc:2379] T 4042b507d68741398ce131a4ebc5bba7 P bb46349f489247b4973643a309634d45 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c88a7a11405f4a7699f9573f7dcfd4c0 in current term 1: Already voted for candidate bb46349f489247b4973643a309634d45 in this term.
I20260810 11:30:16.442418 9492 leader_election.cc:302] T 4042b507d68741398ce131a4ebc5bba7 P c88a7a11405f4a7699f9573f7dcfd4c0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90566adbea5742dea2a7249d96470451, c88a7a11405f4a7699f9573f7dcfd4c0; no voters:
I20260810 11:30:16.443001 9921 raft_consensus.cc:2781] T 4042b507d68741398ce131a4ebc5bba7 P c88a7a11405f4a7699f9573f7dcfd4c0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 11:30:16.443207 9921 raft_consensus.cc:482] T 4042b507d68741398ce131a4ebc5bba7 P c88a7a11405f4a7699f9573f7dcfd4c0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 11:30:16.443338 9921 raft_consensus.cc:3037] T 4042b507d68741398ce131a4ebc5bba7 P c88a7a11405f4a7699f9573f7dcfd4c0 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:30:16.444402 9924 leader_election.cc:288] T 4042b507d68741398ce131a4ebc5bba7 P bb46349f489247b4973643a309634d45 [CANDIDATE]: Term 1 election: Requested vote from peers c88a7a11405f4a7699f9573f7dcfd4c0 (127.8.18.193:39295), 90566adbea5742dea2a7249d96470451 (127.8.18.195:41353)
I20260810 11:30:16.445623 9562 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4042b507d68741398ce131a4ebc5bba7" candidate_uuid: "bb46349f489247b4973643a309634d45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c88a7a11405f4a7699f9573f7dcfd4c0"
I20260810 11:30:16.446472 9843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4042b507d68741398ce131a4ebc5bba7" candidate_uuid: "bb46349f489247b4973643a309634d45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90566adbea5742dea2a7249d96470451"
I20260810 11:30:16.446779 9843 raft_consensus.cc:3037] T 4042b507d68741398ce131a4ebc5bba7 P 90566adbea5742dea2a7249d96470451 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:30:16.448206 9921 raft_consensus.cc:504] T 4042b507d68741398ce131a4ebc5bba7 P c88a7a11405f4a7699f9573f7dcfd4c0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c88a7a11405f4a7699f9573f7dcfd4c0" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 39295 } } peers { permanent_uuid: "bb46349f489247b4973643a309634d45" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 44751 } } peers { permanent_uuid: "90566adbea5742dea2a7249d96470451" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 41353 } }
I20260810 11:30:16.448794 9562 raft_consensus.cc:2379] T 4042b507d68741398ce131a4ebc5bba7 P c88a7a11405f4a7699f9573f7dcfd4c0 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate bb46349f489247b4973643a309634d45 in current term 1: Already voted for candidate c88a7a11405f4a7699f9573f7dcfd4c0 in this term.
I20260810 11:30:16.449620 9843 raft_consensus.cc:2454] T 4042b507d68741398ce131a4ebc5bba7 P 90566adbea5742dea2a7249d96470451 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bb46349f489247b4973643a309634d45 in term 1.
I20260810 11:30:16.449831 9702 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4042b507d68741398ce131a4ebc5bba7" candidate_uuid: "c88a7a11405f4a7699f9573f7dcfd4c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb46349f489247b4973643a309634d45"
I20260810 11:30:16.450419 9633 leader_election.cc:302] T 4042b507d68741398ce131a4ebc5bba7 P bb46349f489247b4973643a309634d45 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90566adbea5742dea2a7249d96470451, bb46349f489247b4973643a309634d45; no voters:
I20260810 11:30:16.450493 9921 leader_election.cc:288] T 4042b507d68741398ce131a4ebc5bba7 P c88a7a11405f4a7699f9573f7dcfd4c0 [CANDIDATE]: Term 1 election: Requested vote from peers bb46349f489247b4973643a309634d45 (127.8.18.194:44751), 90566adbea5742dea2a7249d96470451 (127.8.18.195:41353)
I20260810 11:30:16.450837 9702 raft_consensus.cc:2379] T 4042b507d68741398ce131a4ebc5bba7 P bb46349f489247b4973643a309634d45 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate c88a7a11405f4a7699f9573f7dcfd4c0 in current term 1: Already voted for candidate bb46349f489247b4973643a309634d45 in this term.
I20260810 11:30:16.451081 9924 raft_consensus.cc:2781] T 4042b507d68741398ce131a4ebc5bba7 P bb46349f489247b4973643a309634d45 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:30:16.451498 9843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4042b507d68741398ce131a4ebc5bba7" candidate_uuid: "c88a7a11405f4a7699f9573f7dcfd4c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90566adbea5742dea2a7249d96470451"
I20260810 11:30:16.451766 9924 raft_consensus.cc:686] T 4042b507d68741398ce131a4ebc5bba7 P bb46349f489247b4973643a309634d45 [term 1 LEADER]: Becoming Leader. State: Replica: bb46349f489247b4973643a309634d45, State: Running, Role: LEADER
I20260810 11:30:16.451885 9843 raft_consensus.cc:2379] T 4042b507d68741398ce131a4ebc5bba7 P 90566adbea5742dea2a7249d96470451 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate c88a7a11405f4a7699f9573f7dcfd4c0 in current term 1: Already voted for candidate bb46349f489247b4973643a309634d45 in this term.
I20260810 11:30:16.452107 9924 consensus_queue.cc:237] T 4042b507d68741398ce131a4ebc5bba7 P bb46349f489247b4973643a309634d45 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c88a7a11405f4a7699f9573f7dcfd4c0" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 39295 } } peers { permanent_uuid: "bb46349f489247b4973643a309634d45" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 44751 } } peers { permanent_uuid: "90566adbea5742dea2a7249d96470451" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 41353 } }
I20260810 11:30:16.452589 9492 leader_election.cc:302] T 4042b507d68741398ce131a4ebc5bba7 P c88a7a11405f4a7699f9573f7dcfd4c0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c88a7a11405f4a7699f9573f7dcfd4c0; no voters: 90566adbea5742dea2a7249d96470451, bb46349f489247b4973643a309634d45
I20260810 11:30:16.452973 9921 raft_consensus.cc:2726] T 4042b507d68741398ce131a4ebc5bba7 P c88a7a11405f4a7699f9573f7dcfd4c0 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260810 11:30:16.456017 9401 catalog_manager.cc:5495] T 4042b507d68741398ce131a4ebc5bba7 P bb46349f489247b4973643a309634d45 reported cstate change: term changed from 0 to 1, leader changed from <none> to bb46349f489247b4973643a309634d45 (127.8.18.194). New cstate: current_term: 1 leader_uuid: "bb46349f489247b4973643a309634d45" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c88a7a11405f4a7699f9573f7dcfd4c0" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 39295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bb46349f489247b4973643a309634d45" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 44751 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90566adbea5742dea2a7249d96470451" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 41353 } health_report { overall_health: UNKNOWN } } }
Aug 10 11:30:16 dist-test-slave-srgv krb5kdc[9356](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361413, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.254@KRBTEST.COM for kudu/127.8.18.254@KRBTEST.COM
I20260810 11:30:16.546480 9954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:16.526639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:35405 (local address 127.0.0.1:58032)
0810 11:30:16.528285 (+ 1646us) negotiation.cc:107] Waiting for socket to connect
0810 11:30:16.528303 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 11:30:16.528456 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:30:16.528967 (+ 511us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:30:16.528970 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:30:16.529934 (+ 964us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:30:16.530292 (+ 358us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:30:16.530304 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:30:16.532047 (+ 1743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:30:16.532051 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:30:16.533179 (+ 1128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:30:16.533193 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:30:16.534395 (+ 1202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:30:16.534400 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:30:16.534546 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:16.535386 (+ 840us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:30:16.535405 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:30:16.538297 (+ 2892us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:30:16.543220 (+ 4923us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:30:16.543237 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:30:16.543244 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:30:16.543926 (+ 682us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:30:16.544394 (+ 468us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:30:16.544398 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:30:16.544400 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:30:16.544564 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:30:16.545214 (+ 650us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:30:16.545221 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:30:16.545479 (+ 258us) client_negotiation.cc:769] Sending connection context
0810 11:30:16.545733 (+ 254us) client_negotiation.cc:240] Negotiation successful
0810 11:30:16.546015 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1484,"thread_start_us":1315,"threads_started":1}
I20260810 11:30:16.546739 9953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:16.526644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58032 (local address 127.8.18.254:35405)
0810 11:30:16.527612 (+ 968us) server_negotiation.cc:206] Beginning negotiation
0810 11:30:16.527622 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 11:30:16.528436 (+ 814us) server_negotiation.cc:407] Connection header received
0810 11:30:16.528596 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:30:16.528602 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:30:16.528686 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:30:16.528818 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:30:16.530496 (+ 1678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:16.531862 (+ 1366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:16.533347 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:16.534179 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:16.534317 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:16.538595 (+ 4278us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:30:16.538604 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:30:16.538611 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:30:16.538657 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:30:16.542991 (+ 4334us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:16.544093 (+ 1102us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:16.544099 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:16.544103 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:16.544274 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:16.544682 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:16.544691 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:16.544695 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:16.545074 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:30:16.545254 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0810 11:30:16.545819 (+ 565us) server_negotiation.cc:299] Negotiation successful
0810 11:30:16.546252 (+ 433us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":122,"server-negotiator.queue_time_us":752,"thread_start_us":537,"threads_started":1}
I20260810 11:30:16.553651 9401 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.8.18.254@KRBTEST.COM'} at 127.0.0.1:58032:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260810 11:30:16.555105 9401 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 11:30:16.563417 9540 tablet_service.cc:1467] Processing CreateTablet for tablet f2beb35abf5b40c492747c0f9ad927bb (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=f9fc5f2c284d411389603d995fc73224]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 11:30:16.563517 9681 tablet_service.cc:1467] Processing CreateTablet for tablet f2beb35abf5b40c492747c0f9ad927bb (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=f9fc5f2c284d411389603d995fc73224]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 11:30:16.564133 9681 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f2beb35abf5b40c492747c0f9ad927bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:30:16.564126 9540 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f2beb35abf5b40c492747c0f9ad927bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:30:16.567523 9823 tablet_service.cc:1467] Processing CreateTablet for tablet f2beb35abf5b40c492747c0f9ad927bb (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=f9fc5f2c284d411389603d995fc73224]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 11:30:16.568284 9823 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f2beb35abf5b40c492747c0f9ad927bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:30:16.573478 9915 tablet_bootstrap.cc:492] T f2beb35abf5b40c492747c0f9ad927bb P c88a7a11405f4a7699f9573f7dcfd4c0: Bootstrap starting.
I20260810 11:30:16.573452 9914 tablet_bootstrap.cc:492] T f2beb35abf5b40c492747c0f9ad927bb P bb46349f489247b4973643a309634d45: Bootstrap starting.
I20260810 11:30:16.576018 9915 tablet_bootstrap.cc:654] T f2beb35abf5b40c492747c0f9ad927bb P c88a7a11405f4a7699f9573f7dcfd4c0: Neither blocks nor log segments found. Creating new log.
I20260810 11:30:16.576046 9914 tablet_bootstrap.cc:654] T f2beb35abf5b40c492747c0f9ad927bb P bb46349f489247b4973643a309634d45: Neither blocks nor log segments found. Creating new log.
I20260810 11:30:16.578508 9913 tablet_bootstrap.cc:492] T f2beb35abf5b40c492747c0f9ad927bb P 90566adbea5742dea2a7249d96470451: Bootstrap starting.
I20260810 11:30:16.580541 9915 tablet_bootstrap.cc:492] T f2beb35abf5b40c492747c0f9ad927bb P c88a7a11405f4a7699f9573f7dcfd4c0: No bootstrap required, opened a new log
I20260810 11:30:16.581079 9913 tablet_bootstrap.cc:654] T f2beb35abf5b40c492747c0f9ad927bb P 90566adbea5742dea2a7249d96470451: Neither blocks nor log segments found. Creating new log.
I20260810 11:30:16.582139 9915 ts_tablet_manager.cc:1397] T f2beb35abf5b40c492747c0f9ad927bb P c88a7a11405f4a7699f9573f7dcfd4c0: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.004s
I20260810 11:30:16.582305 9914 tablet_bootstrap.cc:492] T f2beb35abf5b40c492747c0f9ad927bb P bb46349f489247b4973643a309634d45: No bootstrap required, opened a new log
I20260810 11:30:16.582615 9914 ts_tablet_manager.cc:1397] T f2beb35abf5b40c492747c0f9ad927bb P bb46349f489247b4973643a309634d45: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.000s
I20260810 11:30:16.583071 9915 raft_consensus.cc:348] T f2beb35abf5b40c492747c0f9ad927bb P c88a7a11405f4a7699f9573f7dcfd4c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c88a7a11405f4a7699f9573f7dcfd4c0" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 39295 } } peers { permanent_uuid: "bb46349f489247b4973643a309634d45" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 44751 } } peers { permanent_uuid: "90566adbea5742dea2a7249d96470451" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 41353 } }
I20260810 11:30:16.583463 9915 raft_consensus.cc:374] T f2beb35abf5b40c492747c0f9ad927bb P c88a7a11405f4a7699f9573f7dcfd4c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:30:16.583572 9915 raft_consensus.cc:729] T f2beb35abf5b40c492747c0f9ad927bb P c88a7a11405f4a7699f9573f7dcfd4c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c88a7a11405f4a7699f9573f7dcfd4c0, State: Initialized, Role: FOLLOWER
I20260810 11:30:16.583880 9915 consensus_queue.cc:260] T f2beb35abf5b40c492747c0f9ad927bb P c88a7a11405f4a7699f9573f7dcfd4c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c88a7a11405f4a7699f9573f7dcfd4c0" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 39295 } } peers { permanent_uuid: "bb46349f489247b4973643a309634d45" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 44751 } } peers { permanent_uuid: "90566adbea5742dea2a7249d96470451" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 41353 } }
I20260810 11:30:16.584761 9913 tablet_bootstrap.cc:492] T f2beb35abf5b40c492747c0f9ad927bb P 90566adbea5742dea2a7249d96470451: No bootstrap required, opened a new log
I20260810 11:30:16.585005 9913 ts_tablet_manager.cc:1397] T f2beb35abf5b40c492747c0f9ad927bb P 90566adbea5742dea2a7249d96470451: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.004s
I20260810 11:30:16.584931 9914 raft_consensus.cc:348] T f2beb35abf5b40c492747c0f9ad927bb P bb46349f489247b4973643a309634d45 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c88a7a11405f4a7699f9573f7dcfd4c0" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 39295 } } peers { permanent_uuid: "bb46349f489247b4973643a309634d45" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 44751 } } peers { permanent_uuid: "90566adbea5742dea2a7249d96470451" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 41353 } }
I20260810 11:30:16.585274 9914 raft_consensus.cc:374] T f2beb35abf5b40c492747c0f9ad927bb P bb46349f489247b4973643a309634d45 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:30:16.585389 9914 raft_consensus.cc:729] T f2beb35abf5b40c492747c0f9ad927bb P bb46349f489247b4973643a309634d45 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb46349f489247b4973643a309634d45, State: Initialized, Role: FOLLOWER
I20260810 11:30:16.585732 9921 tablet_replica.cc:439] T f2beb35abf5b40c492747c0f9ad927bb P c88a7a11405f4a7699f9573f7dcfd4c0: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c88a7a11405f4a7699f9573f7dcfd4c0" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 39295 } } peers { permanent_uuid: "bb46349f489247b4973643a309634d45" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 44751 } } peers { permanent_uuid: "90566adbea5742dea2a7249d96470451" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 41353 } } }
I20260810 11:30:16.585805 9914 consensus_queue.cc:260] T f2beb35abf5b40c492747c0f9ad927bb P bb46349f489247b4973643a309634d45 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c88a7a11405f4a7699f9573f7dcfd4c0" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 39295 } } peers { permanent_uuid: "bb46349f489247b4973643a309634d45" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 44751 } } peers { permanent_uuid: "90566adbea5742dea2a7249d96470451" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 41353 } }
I20260810 11:30:16.586067 9921 tablet_replica.cc:442] T f2beb35abf5b40c492747c0f9ad927bb P c88a7a11405f4a7699f9573f7dcfd4c0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 11:30:16.586467 9924 tablet_replica.cc:439] T f2beb35abf5b40c492747c0f9ad927bb P bb46349f489247b4973643a309634d45: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c88a7a11405f4a7699f9573f7dcfd4c0" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 39295 } } peers { permanent_uuid: "bb46349f489247b4973643a309634d45" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 44751 } } peers { permanent_uuid: "90566adbea5742dea2a7249d96470451" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 41353 } } }
I20260810 11:30:16.586839 9924 tablet_replica.cc:442] T f2beb35abf5b40c492747c0f9ad927bb P bb46349f489247b4973643a309634d45: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 11:30:16.587311 9913 raft_consensus.cc:348] T f2beb35abf5b40c492747c0f9ad927bb P 90566adbea5742dea2a7249d96470451 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c88a7a11405f4a7699f9573f7dcfd4c0" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 39295 } } peers { permanent_uuid: "bb46349f489247b4973643a309634d45" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 44751 } } peers { permanent_uuid: "90566adbea5742dea2a7249d96470451" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 41353 } }
I20260810 11:30:16.587654 9913 raft_consensus.cc:374] T f2beb35abf5b40c492747c0f9ad927bb P 90566adbea5742dea2a7249d96470451 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:30:16.587922 9913 raft_consensus.cc:729] T f2beb35abf5b40c492747c0f9ad927bb P 90566adbea5742dea2a7249d96470451 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90566adbea5742dea2a7249d96470451, State: Initialized, Role: FOLLOWER
I20260810 11:30:16.588260 9913 consensus_queue.cc:260] T f2beb35abf5b40c492747c0f9ad927bb P 90566adbea5742dea2a7249d96470451 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c88a7a11405f4a7699f9573f7dcfd4c0" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 39295 } } peers { permanent_uuid: "bb46349f489247b4973643a309634d45" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 44751 } } peers { permanent_uuid: "90566adbea5742dea2a7249d96470451" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 41353 } }
I20260810 11:30:16.588949 9913 ts_tablet_manager.cc:1428] T f2beb35abf5b40c492747c0f9ad927bb P 90566adbea5742dea2a7249d96470451: Time spent starting tablet: real 0.004s user 0.000s sys 0.002s
I20260810 11:30:16.588822 9919 tablet_replica.cc:439] T f2beb35abf5b40c492747c0f9ad927bb P 90566adbea5742dea2a7249d96470451: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c88a7a11405f4a7699f9573f7dcfd4c0" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 39295 } } peers { permanent_uuid: "bb46349f489247b4973643a309634d45" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 44751 } } peers { permanent_uuid: "90566adbea5742dea2a7249d96470451" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 41353 } } }
I20260810 11:30:16.589197 9919 tablet_replica.cc:442] T f2beb35abf5b40c492747c0f9ad927bb P 90566adbea5742dea2a7249d96470451: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 11:30:16.586493 9914 ts_tablet_manager.cc:1428] T f2beb35abf5b40c492747c0f9ad927bb P bb46349f489247b4973643a309634d45: Time spent starting tablet: real 0.004s user 0.000s sys 0.002s
I20260810 11:30:16.592402 9915 ts_tablet_manager.cc:1428] T f2beb35abf5b40c492747c0f9ad927bb P c88a7a11405f4a7699f9573f7dcfd4c0: Time spent starting tablet: real 0.010s user 0.000s sys 0.002s
I20260810 11:30:16.832834 9924 raft_consensus.cc:482] T f2beb35abf5b40c492747c0f9ad927bb P bb46349f489247b4973643a309634d45 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 11:30:16.833082 9924 raft_consensus.cc:504] T f2beb35abf5b40c492747c0f9ad927bb P bb46349f489247b4973643a309634d45 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c88a7a11405f4a7699f9573f7dcfd4c0" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 39295 } } peers { permanent_uuid: "bb46349f489247b4973643a309634d45" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 44751 } } peers { permanent_uuid: "90566adbea5742dea2a7249d96470451" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 41353 } }
I20260810 11:30:16.833843 9924 leader_election.cc:288] T f2beb35abf5b40c492747c0f9ad927bb P bb46349f489247b4973643a309634d45 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c88a7a11405f4a7699f9573f7dcfd4c0 (127.8.18.193:39295), 90566adbea5742dea2a7249d96470451 (127.8.18.195:41353)
I20260810 11:30:16.834584 9843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f2beb35abf5b40c492747c0f9ad927bb" candidate_uuid: "bb46349f489247b4973643a309634d45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90566adbea5742dea2a7249d96470451" is_pre_election: true
I20260810 11:30:16.834586 9562 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f2beb35abf5b40c492747c0f9ad927bb" candidate_uuid: "bb46349f489247b4973643a309634d45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c88a7a11405f4a7699f9573f7dcfd4c0" is_pre_election: true
I20260810 11:30:16.834849 9562 raft_consensus.cc:2454] T f2beb35abf5b40c492747c0f9ad927bb P c88a7a11405f4a7699f9573f7dcfd4c0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bb46349f489247b4973643a309634d45 in term 0.
I20260810 11:30:16.834879 9843 raft_consensus.cc:2454] T f2beb35abf5b40c492747c0f9ad927bb P 90566adbea5742dea2a7249d96470451 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bb46349f489247b4973643a309634d45 in term 0.
I20260810 11:30:16.835354 9631 leader_election.cc:302] T f2beb35abf5b40c492747c0f9ad927bb P bb46349f489247b4973643a309634d45 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bb46349f489247b4973643a309634d45, c88a7a11405f4a7699f9573f7dcfd4c0; no voters:
I20260810 11:30:16.835685 9924 raft_consensus.cc:2781] T f2beb35abf5b40c492747c0f9ad927bb P bb46349f489247b4973643a309634d45 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 11:30:16.835812 9924 raft_consensus.cc:482] T f2beb35abf5b40c492747c0f9ad927bb P bb46349f489247b4973643a309634d45 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 11:30:16.835889 9924 raft_consensus.cc:3037] T f2beb35abf5b40c492747c0f9ad927bb P bb46349f489247b4973643a309634d45 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:30:16.838222 9924 raft_consensus.cc:504] T f2beb35abf5b40c492747c0f9ad927bb P bb46349f489247b4973643a309634d45 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c88a7a11405f4a7699f9573f7dcfd4c0" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 39295 } } peers { permanent_uuid: "bb46349f489247b4973643a309634d45" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 44751 } } peers { permanent_uuid: "90566adbea5742dea2a7249d96470451" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 41353 } }
I20260810 11:30:16.838977 9924 leader_election.cc:288] T f2beb35abf5b40c492747c0f9ad927bb P bb46349f489247b4973643a309634d45 [CANDIDATE]: Term 1 election: Requested vote from peers c88a7a11405f4a7699f9573f7dcfd4c0 (127.8.18.193:39295), 90566adbea5742dea2a7249d96470451 (127.8.18.195:41353)
I20260810 11:30:16.839493 9562 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f2beb35abf5b40c492747c0f9ad927bb" candidate_uuid: "bb46349f489247b4973643a309634d45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c88a7a11405f4a7699f9573f7dcfd4c0"
I20260810 11:30:16.839516 9843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f2beb35abf5b40c492747c0f9ad927bb" candidate_uuid: "bb46349f489247b4973643a309634d45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90566adbea5742dea2a7249d96470451"
I20260810 11:30:16.839757 9562 raft_consensus.cc:3037] T f2beb35abf5b40c492747c0f9ad927bb P c88a7a11405f4a7699f9573f7dcfd4c0 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:30:16.839782 9843 raft_consensus.cc:3037] T f2beb35abf5b40c492747c0f9ad927bb P 90566adbea5742dea2a7249d96470451 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:30:16.842082 9562 raft_consensus.cc:2454] T f2beb35abf5b40c492747c0f9ad927bb P c88a7a11405f4a7699f9573f7dcfd4c0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bb46349f489247b4973643a309634d45 in term 1.
I20260810 11:30:16.842657 9631 leader_election.cc:302] T f2beb35abf5b40c492747c0f9ad927bb P bb46349f489247b4973643a309634d45 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bb46349f489247b4973643a309634d45, c88a7a11405f4a7699f9573f7dcfd4c0; no voters:
I20260810 11:30:16.842911 9843 raft_consensus.cc:2454] T f2beb35abf5b40c492747c0f9ad927bb P 90566adbea5742dea2a7249d96470451 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bb46349f489247b4973643a309634d45 in term 1.
I20260810 11:30:16.842998 9924 raft_consensus.cc:2781] T f2beb35abf5b40c492747c0f9ad927bb P bb46349f489247b4973643a309634d45 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:30:16.843408 9924 raft_consensus.cc:686] T f2beb35abf5b40c492747c0f9ad927bb P bb46349f489247b4973643a309634d45 [term 1 LEADER]: Becoming Leader. State: Replica: bb46349f489247b4973643a309634d45, State: Running, Role: LEADER
I20260810 11:30:16.843779 9924 consensus_queue.cc:237] T f2beb35abf5b40c492747c0f9ad927bb P bb46349f489247b4973643a309634d45 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c88a7a11405f4a7699f9573f7dcfd4c0" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 39295 } } peers { permanent_uuid: "bb46349f489247b4973643a309634d45" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 44751 } } peers { permanent_uuid: "90566adbea5742dea2a7249d96470451" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 41353 } }
I20260810 11:30:16.845713 9920 tablet_replica.cc:439] T f2beb35abf5b40c492747c0f9ad927bb P bb46349f489247b4973643a309634d45: TxnStatusTablet state changed. Reason: New leader bb46349f489247b4973643a309634d45. Latest consensus state: current_term: 1 leader_uuid: "bb46349f489247b4973643a309634d45" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c88a7a11405f4a7699f9573f7dcfd4c0" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 39295 } } peers { permanent_uuid: "bb46349f489247b4973643a309634d45" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 44751 } } peers { permanent_uuid: "90566adbea5742dea2a7249d96470451" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 41353 } } }
I20260810 11:30:16.846047 9920 tablet_replica.cc:442] T f2beb35abf5b40c492747c0f9ad927bb P bb46349f489247b4973643a309634d45: This TxnStatusTablet replica's current role is: LEADER
I20260810 11:30:16.846791 9960 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260810 11:30:16.847239 9960 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260810 11:30:16.847658 9399 catalog_manager.cc:5495] T f2beb35abf5b40c492747c0f9ad927bb P bb46349f489247b4973643a309634d45 reported cstate change: term changed from 0 to 1, leader changed from <none> to bb46349f489247b4973643a309634d45 (127.8.18.194). New cstate: current_term: 1 leader_uuid: "bb46349f489247b4973643a309634d45" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c88a7a11405f4a7699f9573f7dcfd4c0" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 39295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bb46349f489247b4973643a309634d45" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 44751 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90566adbea5742dea2a7249d96470451" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 41353 } health_report { overall_health: UNKNOWN } } }
I20260810 11:30:16.873960 9920 consensus_queue.cc:1035] T 4042b507d68741398ce131a4ebc5bba7 P bb46349f489247b4973643a309634d45 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90566adbea5742dea2a7249d96470451" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 41353 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 11:30:16.881448 9924 consensus_queue.cc:1035] T 4042b507d68741398ce131a4ebc5bba7 P bb46349f489247b4973643a309634d45 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c88a7a11405f4a7699f9573f7dcfd4c0" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 39295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 11:30:16.904537 9911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:16.896494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50290 (local address 127.8.18.194:44751)
0810 11:30:16.896731 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0810 11:30:16.896737 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:30:16.896945 (+ 208us) server_negotiation.cc:407] Connection header received
0810 11:30:16.897218 (+ 273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:30:16.897221 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:30:16.897342 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:30:16.897502 (+ 160us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 11:30:16.898002 (+ 500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:16.899184 (+ 1182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:16.901900 (+ 2716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:16.902553 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:16.902682 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:16.903585 (+ 903us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 11:30:16.903712 (+ 127us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 11:30:16.903800 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0810 11:30:16.904077 (+ 277us) server_negotiation.cc:299] Negotiation successful
0810 11:30:16.904168 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260810 11:30:16.904745 9954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:16.896556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.194:44751 (local address 127.0.0.1:50290)
0810 11:30:16.896843 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0810 11:30:16.896861 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 11:30:16.897017 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:30:16.897539 (+ 522us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:30:16.897559 (+ 20us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:30:16.897577 (+ 18us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 11:30:16.897852 (+ 275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:30:16.897866 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:30:16.899363 (+ 1497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:30:16.899369 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:30:16.901683 (+ 2314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:30:16.901711 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:30:16.902806 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:30:16.902814 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:30:16.903124 (+ 310us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:16.903267 (+ 143us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 11:30:16.903851 (+ 584us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 11:30:16.903881 (+ 30us) client_negotiation.cc:769] Sending connection context
0810 11:30:16.903986 (+ 105us) client_negotiation.cc:240] Negotiation successful
0810 11:30:16.904234 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":68}
I20260810 11:30:16.910638 9562 raft_consensus.cc:1261] T f2beb35abf5b40c492747c0f9ad927bb P c88a7a11405f4a7699f9573f7dcfd4c0 [term 1 FOLLOWER]: Refusing update from remote peer bb46349f489247b4973643a309634d45: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 11:30:16.910617 9843 raft_consensus.cc:1261] T f2beb35abf5b40c492747c0f9ad927bb P 90566adbea5742dea2a7249d96470451 [term 1 FOLLOWER]: Refusing update from remote peer bb46349f489247b4973643a309634d45: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 11:30:16.911538 9963 consensus_queue.cc:1035] T f2beb35abf5b40c492747c0f9ad927bb P bb46349f489247b4973643a309634d45 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c88a7a11405f4a7699f9573f7dcfd4c0" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 39295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260810 11:30:16.912218 9920 consensus_queue.cc:1035] T f2beb35abf5b40c492747c0f9ad927bb P bb46349f489247b4973643a309634d45 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90566adbea5742dea2a7249d96470451" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 41353 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 11:30:16.917794 9919 tablet_replica.cc:439] T f2beb35abf5b40c492747c0f9ad927bb P 90566adbea5742dea2a7249d96470451: TxnStatusTablet state changed. Reason: New leader bb46349f489247b4973643a309634d45. Latest consensus state: current_term: 1 leader_uuid: "bb46349f489247b4973643a309634d45" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c88a7a11405f4a7699f9573f7dcfd4c0" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 39295 } } peers { permanent_uuid: "bb46349f489247b4973643a309634d45" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 44751 } } peers { permanent_uuid: "90566adbea5742dea2a7249d96470451" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 41353 } } }
I20260810 11:30:16.918227 9919 tablet_replica.cc:442] T f2beb35abf5b40c492747c0f9ad927bb P 90566adbea5742dea2a7249d96470451: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 11:30:16.922538 9964 mvcc.cc:205] Tried to move back new op lower bound from 7316936363656695808 to 7316936363394809856. Current Snapshot: MvccSnapshot[applied={T|T < 7316936363656695808}]
I20260810 11:30:16.923166 9921 tablet_replica.cc:439] T f2beb35abf5b40c492747c0f9ad927bb P c88a7a11405f4a7699f9573f7dcfd4c0: TxnStatusTablet state changed. Reason: New leader bb46349f489247b4973643a309634d45. Latest consensus state: current_term: 1 leader_uuid: "bb46349f489247b4973643a309634d45" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c88a7a11405f4a7699f9573f7dcfd4c0" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 39295 } } peers { permanent_uuid: "bb46349f489247b4973643a309634d45" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 44751 } } peers { permanent_uuid: "90566adbea5742dea2a7249d96470451" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 41353 } } }
I20260810 11:30:16.923586 9921 tablet_replica.cc:442] T f2beb35abf5b40c492747c0f9ad927bb P c88a7a11405f4a7699f9573f7dcfd4c0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 11:30:16.928583 9921 tablet_replica.cc:439] T f2beb35abf5b40c492747c0f9ad927bb P c88a7a11405f4a7699f9573f7dcfd4c0: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "bb46349f489247b4973643a309634d45" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c88a7a11405f4a7699f9573f7dcfd4c0" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 39295 } } peers { permanent_uuid: "bb46349f489247b4973643a309634d45" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 44751 } } peers { permanent_uuid: "90566adbea5742dea2a7249d96470451" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 41353 } } }
I20260810 11:30:16.928977 9921 tablet_replica.cc:442] T f2beb35abf5b40c492747c0f9ad927bb P c88a7a11405f4a7699f9573f7dcfd4c0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 11:30:16.929198 9920 tablet_replica.cc:439] T f2beb35abf5b40c492747c0f9ad927bb P bb46349f489247b4973643a309634d45: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "bb46349f489247b4973643a309634d45" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c88a7a11405f4a7699f9573f7dcfd4c0" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 39295 } } peers { permanent_uuid: "bb46349f489247b4973643a309634d45" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 44751 } } peers { permanent_uuid: "90566adbea5742dea2a7249d96470451" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 41353 } } }
I20260810 11:30:16.930925 9920 tablet_replica.cc:442] T f2beb35abf5b40c492747c0f9ad927bb P bb46349f489247b4973643a309634d45: This TxnStatusTablet replica's current role is: LEADER
I20260810 11:30:16.928181 9963 tablet_replica.cc:439] T f2beb35abf5b40c492747c0f9ad927bb P bb46349f489247b4973643a309634d45: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "bb46349f489247b4973643a309634d45" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c88a7a11405f4a7699f9573f7dcfd4c0" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 39295 } } peers { permanent_uuid: "bb46349f489247b4973643a309634d45" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 44751 } } peers { permanent_uuid: "90566adbea5742dea2a7249d96470451" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 41353 } } }
I20260810 11:30:16.932446 9963 tablet_replica.cc:442] T f2beb35abf5b40c492747c0f9ad927bb P bb46349f489247b4973643a309634d45: This TxnStatusTablet replica's current role is: LEADER
I20260810 11:30:16.943012 9919 tablet_replica.cc:439] T f2beb35abf5b40c492747c0f9ad927bb P 90566adbea5742dea2a7249d96470451: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "bb46349f489247b4973643a309634d45" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c88a7a11405f4a7699f9573f7dcfd4c0" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 39295 } } peers { permanent_uuid: "bb46349f489247b4973643a309634d45" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 44751 } } peers { permanent_uuid: "90566adbea5742dea2a7249d96470451" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 41353 } } }
I20260810 11:30:16.943495 9919 tablet_replica.cc:442] T f2beb35abf5b40c492747c0f9ad927bb P 90566adbea5742dea2a7249d96470451: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 11:30:16.962023 9911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:16.954795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.194:58133 (local address 127.8.18.194:44751)
0810 11:30:16.954973 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0810 11:30:16.954977 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 11:30:16.956125 (+ 1148us) server_negotiation.cc:407] Connection header received
0810 11:30:16.956315 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:30:16.956321 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:30:16.956407 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:30:16.956513 (+ 106us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 11:30:16.957302 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:16.958193 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:16.959848 (+ 1655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:16.960393 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:16.960547 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:16.961061 (+ 514us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 11:30:16.961204 (+ 143us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 11:30:16.961314 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0810 11:30:16.961517 (+ 203us) server_negotiation.cc:299] Negotiation successful
0810 11:30:16.961587 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260810 11:30:16.962512 9975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:16.954849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.194:44751 (local address 127.8.18.194:58133)
0810 11:30:16.956003 (+ 1154us) negotiation.cc:107] Waiting for socket to connect
0810 11:30:16.956021 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 11:30:16.956183 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:30:16.956533 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:30:16.956536 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:30:16.956546 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 11:30:16.957133 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:30:16.957146 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:30:16.958342 (+ 1196us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:30:16.958349 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:30:16.959672 (+ 1323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:30:16.959686 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:30:16.960622 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:30:16.960630 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:30:16.960806 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:16.960872 (+ 66us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 11:30:16.961314 (+ 442us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 11:30:16.961345 (+ 31us) client_negotiation.cc:769] Sending connection context
0810 11:30:16.961488 (+ 143us) client_negotiation.cc:240] Negotiation successful
0810 11:30:16.961632 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":797,"thread_start_us":504,"threads_started":1}
I20260810 11:30:17.049180 9399 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48576:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 11:30:17.049446 9399 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48576:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 11:30:17.053009 9399 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 41a39e4b02e143cda20aa5740d974b27: Sending DeleteTablet for 3 replicas of tablet 4042b507d68741398ce131a4ebc5bba7
I20260810 11:30:17.054066 9540 tablet_service.cc:1514] Processing DeleteTablet for tablet 4042b507d68741398ce131a4ebc5bba7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:30:17 UTC) from {username='kudu', principal='kudu/127.8.18.254@KRBTEST.COM'} at 127.0.0.1:49340
I20260810 11:30:17.054219 9823 tablet_service.cc:1514] Processing DeleteTablet for tablet 4042b507d68741398ce131a4ebc5bba7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:30:17 UTC) from {username='kudu', principal='kudu/127.8.18.254@KRBTEST.COM'} at 127.0.0.1:43176
I20260810 11:30:17.057718 9681 tablet_service.cc:1514] Processing DeleteTablet for tablet 4042b507d68741398ce131a4ebc5bba7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:30:17 UTC) from {username='kudu', principal='kudu/127.8.18.254@KRBTEST.COM'} at 127.0.0.1:35768
I20260810 11:30:17.067034 9979 tablet_replica.cc:330] T 4042b507d68741398ce131a4ebc5bba7 P 90566adbea5742dea2a7249d96470451: stopping tablet replica
I20260810 11:30:17.067509 9978 tablet_replica.cc:330] T 4042b507d68741398ce131a4ebc5bba7 P c88a7a11405f4a7699f9573f7dcfd4c0: stopping tablet replica
I20260810 11:30:17.067710 9979 raft_consensus.cc:2229] T 4042b507d68741398ce131a4ebc5bba7 P 90566adbea5742dea2a7249d96470451 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 11:30:17.067833 9978 raft_consensus.cc:2229] T 4042b507d68741398ce131a4ebc5bba7 P c88a7a11405f4a7699f9573f7dcfd4c0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 11:30:17.068250 9979 raft_consensus.cc:2258] T 4042b507d68741398ce131a4ebc5bba7 P 90566adbea5742dea2a7249d96470451 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:30:17.068336 9978 raft_consensus.cc:2258] T 4042b507d68741398ce131a4ebc5bba7 P c88a7a11405f4a7699f9573f7dcfd4c0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:30:17.068938 9981 tablet_replica.cc:330] T 4042b507d68741398ce131a4ebc5bba7 P bb46349f489247b4973643a309634d45: stopping tablet replica
I20260810 11:30:17.069808 9979 ts_tablet_manager.cc:1905] T 4042b507d68741398ce131a4ebc5bba7 P 90566adbea5742dea2a7249d96470451: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:30:17.069240 9981 raft_consensus.cc:2229] T 4042b507d68741398ce131a4ebc5bba7 P bb46349f489247b4973643a309634d45 [term 1 LEADER]: Raft consensus shutting down.
I20260810 11:30:17.070590 9981 raft_consensus.cc:2258] T 4042b507d68741398ce131a4ebc5bba7 P bb46349f489247b4973643a309634d45 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:30:17.072191 9981 ts_tablet_manager.cc:1905] T 4042b507d68741398ce131a4ebc5bba7 P bb46349f489247b4973643a309634d45: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:30:17.078171 9978 ts_tablet_manager.cc:1905] T 4042b507d68741398ce131a4ebc5bba7 P c88a7a11405f4a7699f9573f7dcfd4c0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:30:17.079411 9979 ts_tablet_manager.cc:1918] T 4042b507d68741398ce131a4ebc5bba7 P 90566adbea5742dea2a7249d96470451: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 11:30:17.079679 9979 log.cc:1196] T 4042b507d68741398ce131a4ebc5bba7 P 90566adbea5742dea2a7249d96470451: Deleting WAL directory at /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786361401922362-8267-0/minicluster-data/ts-2/wal/wals/4042b507d68741398ce131a4ebc5bba7
I20260810 11:30:17.080439 9979 ts_tablet_manager.cc:1939] T 4042b507d68741398ce131a4ebc5bba7 P 90566adbea5742dea2a7249d96470451: Deleting consensus metadata
I20260810 11:30:17.083053 9981 ts_tablet_manager.cc:1918] T 4042b507d68741398ce131a4ebc5bba7 P bb46349f489247b4973643a309634d45: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 11:30:17.083256 9981 log.cc:1196] T 4042b507d68741398ce131a4ebc5bba7 P bb46349f489247b4973643a309634d45: Deleting WAL directory at /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786361401922362-8267-0/minicluster-data/ts-1/wal/wals/4042b507d68741398ce131a4ebc5bba7
I20260810 11:30:17.083992 9981 ts_tablet_manager.cc:1939] T 4042b507d68741398ce131a4ebc5bba7 P bb46349f489247b4973643a309634d45: Deleting consensus metadata
I20260810 11:30:17.085402 9388 catalog_manager.cc:4841] TS bb46349f489247b4973643a309634d45 (127.8.18.194:44751): tablet 4042b507d68741398ce131a4ebc5bba7 (table test-table [id=722cc5a49f6343ea9a77c7f5911f0065]) successfully deleted
I20260810 11:30:17.086935 9387 catalog_manager.cc:4841] TS 90566adbea5742dea2a7249d96470451 (127.8.18.195:41353): tablet 4042b507d68741398ce131a4ebc5bba7 (table test-table [id=722cc5a49f6343ea9a77c7f5911f0065]) successfully deleted
I20260810 11:30:17.089944 9978 ts_tablet_manager.cc:1918] T 4042b507d68741398ce131a4ebc5bba7 P c88a7a11405f4a7699f9573f7dcfd4c0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 11:30:17.090129 9978 log.cc:1196] T 4042b507d68741398ce131a4ebc5bba7 P c88a7a11405f4a7699f9573f7dcfd4c0: Deleting WAL directory at /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786361401922362-8267-0/minicluster-data/ts-0/wal/wals/4042b507d68741398ce131a4ebc5bba7
I20260810 11:30:17.090737 9978 ts_tablet_manager.cc:1939] T 4042b507d68741398ce131a4ebc5bba7 P c88a7a11405f4a7699f9573f7dcfd4c0: Deleting consensus metadata
I20260810 11:30:17.091945 9385 catalog_manager.cc:4841] TS c88a7a11405f4a7699f9573f7dcfd4c0 (127.8.18.193:39295): tablet 4042b507d68741398ce131a4ebc5bba7 (table test-table [id=722cc5a49f6343ea9a77c7f5911f0065]) successfully deleted
Aug 10 11:30:17 dist-test-slave-srgv krb5kdc[9356](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361415, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.8.18.193@KRBTEST.COM
I20260810 11:30:17.107425 9982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:17.069145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54142 (local address 127.8.18.193:39295)
0810 11:30:17.077962 (+ 8817us) server_negotiation.cc:206] Beginning negotiation
0810 11:30:17.077969 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:30:17.081312 (+ 3343us) server_negotiation.cc:407] Connection header received
0810 11:30:17.081583 (+ 271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:30:17.081589 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:30:17.081699 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:30:17.081834 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:30:17.089778 (+ 7944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:17.091153 (+ 1375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:17.093095 (+ 1942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:17.093811 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:17.093974 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:17.099622 (+ 5648us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:30:17.099642 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:30:17.099658 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:30:17.099706 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:30:17.103399 (+ 3693us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:17.104481 (+ 1082us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:17.104490 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:17.104493 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:17.104735 (+ 242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:17.105321 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:17.105326 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:17.105328 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:17.105884 (+ 556us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:30:17.106260 (+ 376us) server_negotiation.cc:1035] Waiting for connection context
0810 11:30:17.106652 (+ 392us) server_negotiation.cc:299] Negotiation successful
0810 11:30:17.106948 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8650,"thread_start_us":10586,"threads_started":1}
W20260810 11:30:17.109057 9606 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54142
I20260810 11:30:17.139928 9985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:17.114321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58040 (local address 127.8.18.254:35405)
0810 11:30:17.115152 (+ 831us) server_negotiation.cc:206] Beginning negotiation
0810 11:30:17.115162 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 11:30:17.122103 (+ 6941us) server_negotiation.cc:407] Connection header received
0810 11:30:17.122356 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:30:17.122366 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:30:17.122442 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:30:17.122571 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:30:17.124509 (+ 1938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:17.125926 (+ 1417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:17.127397 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:17.128082 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:17.128250 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:17.131654 (+ 3404us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:30:17.131660 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:30:17.131664 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:30:17.131738 (+ 74us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:30:17.135398 (+ 3660us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:17.136382 (+ 984us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:17.136397 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:17.136399 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:17.136585 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:17.138017 (+ 1432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:17.138022 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:17.138025 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:17.138407 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:30:17.138653 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0810 11:30:17.139190 (+ 537us) server_negotiation.cc:299] Negotiation successful
0810 11:30:17.139465 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":666,"thread_start_us":525,"threads_started":1}
W20260810 11:30:17.140659 9399 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58040
I20260810 11:30:17.143203 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 9474
I20260810 11:30:17.162060 9982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:17.144333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54156 (local address 127.8.18.193:39295)
0810 11:30:17.144585 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0810 11:30:17.144594 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 11:30:17.145931 (+ 1337us) server_negotiation.cc:407] Connection header received
0810 11:30:17.146149 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:30:17.146154 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:30:17.146226 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:30:17.146318 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:30:17.148323 (+ 2005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:17.149079 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:17.151194 (+ 2115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:17.151652 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:17.151795 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:17.155487 (+ 3692us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:30:17.155492 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:30:17.155496 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:30:17.155540 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:30:17.158736 (+ 3196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:17.159781 (+ 1045us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:17.159786 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:17.159789 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:17.159949 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:17.160433 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:17.160436 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:17.160438 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:17.160714 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:30:17.160924 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0810 11:30:17.161359 (+ 435us) server_negotiation.cc:299] Negotiation successful
0810 11:30:17.161671 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
W20260810 11:30:17.162693 9606 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54156
W20260810 11:30:17.163398 8267 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 9474: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 11:30:17.163620 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 9474
I20260810 11:30:17.183661 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 9615
Aug 10 11:30:17 dist-test-slave-srgv krb5kdc[9356](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361415, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.8.18.194@KRBTEST.COM
I20260810 11:30:17.204509 9911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:17.184573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50294 (local address 127.8.18.194:44751)
0810 11:30:17.184800 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0810 11:30:17.184806 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:30:17.184832 (+ 26us) server_negotiation.cc:407] Connection header received
0810 11:30:17.185057 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:30:17.185065 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:30:17.185166 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:30:17.185295 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:30:17.186947 (+ 1652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:17.188148 (+ 1201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:17.189859 (+ 1711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:17.190593 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:17.190768 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:17.196179 (+ 5411us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:30:17.196204 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:30:17.196228 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:30:17.196291 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:30:17.200802 (+ 4511us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:17.201873 (+ 1071us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:17.201882 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:17.201884 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:17.202062 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:17.202541 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:17.202545 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:17.202547 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:17.202971 (+ 424us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:30:17.203403 (+ 432us) server_negotiation.cc:1035] Waiting for connection context
0810 11:30:17.203763 (+ 360us) server_negotiation.cc:299] Negotiation successful
0810 11:30:17.204096 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
W20260810 11:30:17.205663 9747 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50294
W20260810 11:30:17.206489 8267 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 9615: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 11:30:17.206709 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 9615
I20260810 11:30:17.233083 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 9756
Aug 10 11:30:17 dist-test-slave-srgv krb5kdc[9356](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361415, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.8.18.195@KRBTEST.COM
I20260810 11:30:17.251052 9988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:17.234143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33458 (local address 127.8.18.195:41353)
0810 11:30:17.234663 (+ 520us) server_negotiation.cc:206] Beginning negotiation
0810 11:30:17.234668 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:30:17.234685 (+ 17us) server_negotiation.cc:407] Connection header received
0810 11:30:17.234742 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:30:17.234746 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:30:17.234807 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:30:17.234907 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:30:17.236327 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:17.237039 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:17.238415 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:17.238888 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:17.238992 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:17.243768 (+ 4776us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:30:17.243787 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:30:17.243804 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:30:17.243860 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:30:17.247502 (+ 3642us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:17.248402 (+ 900us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:17.248410 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:17.248412 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:17.248582 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:17.249045 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:17.249050 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:17.249052 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:17.249742 (+ 690us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:30:17.250018 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0810 11:30:17.250401 (+ 383us) server_negotiation.cc:299] Negotiation successful
0810 11:30:17.250633 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":405,"thread_start_us":264,"threads_started":1}
W20260810 11:30:17.251875 9888 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33458
W20260810 11:30:17.252907 8267 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 9756: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 11:30:17.253098 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 9756
I20260810 11:30:17.271692 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 9369
I20260810 11:30:17.273113 9442 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:30:17.394608 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 9369
2026-08-10T11:30:17Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (5373 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786361401922362-8267-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 11:30:17 dist-test-slave-srgv krb5kdc[9992](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 11:30:17 dist-test-slave-srgv krb5kdc[9992](info): set up 2 sockets
Aug 10 11:30:17 dist-test-slave-srgv krb5kdc[9992](info): commencing operation
krb5kdc: starting...
W20260810 11:30:18.524415 8267 mini_kdc.cc:121] Time spent starting KDC: real 1.083s user 0.002s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786361401922362-8267-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786361401922362-8267-0/krb5kdc/test-user.keytab.
Aug 10 11:30:18 dist-test-slave-srgv krb5kdc[9992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361418, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T11:30:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T11:30:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.18.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.18.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.18.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.254.keytab.
Entry for principal kudu/127.8.18.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.254.keytab.
I20260810 11:30:18.753955 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786361401922362-8267-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786361401922362-8267-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786361401922362-8267-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.18.254:40395
--webserver_interface=127.8.18.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.254.keytab
--principal=kudu/127.8.18.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.18.212:43379
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.18.254:40395
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786361401922362-8267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786361401922362-8267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786361401922362-8267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:30:19.119606 10005 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:30:19.120072 10005 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:30:19.120258 10005 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:30:19.129568 10005 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 11:30:19.129756 10005 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260810 11:30:19.129861 10005 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:30:19.129958 10005 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:30:19.130059 10005 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 11:30:19.130153 10005 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 11:30:19.141086 10005 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.18.212:43379
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786361401922362-8267-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786361401922362-8267-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.18.254:40395
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.254.keytab
--principal=kudu/127.8.18.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.18.254:40395
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.18.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786361401922362-8267-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:30:19.142656 10005 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:30:19.144212 10005 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:30:19.157584 10011 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:30:19.157613 10010 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:30:19.157778 10013 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:30:19.158622 10005 server_base.cc:1034] running on GCE node
I20260810 11:30:19.159564 10005 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:30:19.161336 10005 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:30:19.162683 10005 hybrid_clock.cc:648] HybridClock initialized: now 1786361419162587 us; error 72 us; skew 500 ppm
I20260810 11:30:19.163152 10005 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:30:19 dist-test-slave-srgv krb5kdc[9992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361419, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:30:19.172246 10005 init.cc:377] Logged in from keytab as kudu/127.8.18.254@KRBTEST.COM (short username kudu)
I20260810 11:30:19.174204 10005 webserver.cc:460] Webserver started at http://127.8.18.254:33469/ using document root <none> and password file <none>
I20260810 11:30:19.175200 10005 fs_manager.cc:359] Metadata directory not provided
I20260810 11:30:19.175351 10005 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:30:19.175740 10005 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:30:19.179414 10005 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786361401922362-8267-0/minicluster-data/master-0/data/instance:
uuid: "ed0668521a7b4c0e82440311f75dd278"
format_stamp: "Formatted at 2026-08-10 11:30:19 on dist-test-slave-srgv"
server_key: "14e050e691f55d1332f34a1b9ad1988d"
server_key_iv: "f82a1c9f569ee3e10ad44b3f4305fe09"
server_key_version: "encryptionkey@0"
I20260810 11:30:19.180393 10005 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786361401922362-8267-0/minicluster-data/master-0/wal/instance:
uuid: "ed0668521a7b4c0e82440311f75dd278"
format_stamp: "Formatted at 2026-08-10 11:30:19 on dist-test-slave-srgv"
server_key: "14e050e691f55d1332f34a1b9ad1988d"
server_key_iv: "f82a1c9f569ee3e10ad44b3f4305fe09"
server_key_version: "encryptionkey@0"
I20260810 11:30:19.186928 10005 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260810 11:30:19.191736 10020 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:30:19.193593 10005 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260810 11:30:19.193852 10005 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786361401922362-8267-0/minicluster-data/master-0/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786361401922362-8267-0/minicluster-data/master-0/wal
uuid: "ed0668521a7b4c0e82440311f75dd278"
format_stamp: "Formatted at 2026-08-10 11:30:19 on dist-test-slave-srgv"
server_key: "14e050e691f55d1332f34a1b9ad1988d"
server_key_iv: "f82a1c9f569ee3e10ad44b3f4305fe09"
server_key_version: "encryptionkey@0"
I20260810 11:30:19.194166 10005 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786361401922362-8267-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786361401922362-8267-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786361401922362-8267-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:30:19.221329 10005 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:30:19.226563 10005 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:30:19.226891 10005 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:30:19.252396 10005 rpc_server.cc:307] RPC server started. Bound to: 127.8.18.254:40395
I20260810 11:30:19.252436 10083 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.18.254:40395 every 8 connection(s)
I20260810 11:30:19.254114 10005 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
I20260810 11:30:19.257288 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 10005
I20260810 11:30:19.257773 8267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786361401922362-8267-0/minicluster-data/master-0/wal/instance
I20260810 11:30:19.258493 8267 external_mini_cluster.cc:1428] Setting key 3eca7accbbdf773918d96031b0fbb2a7
I20260810 11:30:19.258576 10084 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:30:19.275079 10084 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ed0668521a7b4c0e82440311f75dd278: Bootstrap starting.
Aug 10 11:30:19 dist-test-slave-srgv krb5kdc[9992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361418, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.18.254@KRBTEST.COM
I20260810 11:30:19.279800 10084 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ed0668521a7b4c0e82440311f75dd278: Neither blocks nor log segments found. Creating new log.
I20260810 11:30:19.281553 10084 log.cc:824] T 00000000000000000000000000000000 P ed0668521a7b4c0e82440311f75dd278: Log is configured to *not* fsync() on all Append() calls
I20260810 11:30:19.286089 10084 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ed0668521a7b4c0e82440311f75dd278: No bootstrap required, opened a new log
I20260810 11:30:19.287461 10087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:19.260721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41800 (local address 127.8.18.254:40395)
0810 11:30:19.261647 (+ 926us) server_negotiation.cc:206] Beginning negotiation
0810 11:30:19.261660 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 11:30:19.261705 (+ 45us) server_negotiation.cc:407] Connection header received
0810 11:30:19.263830 (+ 2125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:30:19.263856 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:30:19.264330 (+ 474us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:30:19.264723 (+ 393us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:30:19.266875 (+ 2152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:19.268268 (+ 1393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:19.270247 (+ 1979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:19.271116 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:19.271290 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:19.276708 (+ 5418us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:30:19.276734 (+ 26us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:30:19.276760 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:30:19.276839 (+ 79us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:30:19.281372 (+ 4533us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:19.282688 (+ 1316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:19.282700 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:19.282708 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:19.282951 (+ 243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:19.283577 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:19.283583 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:19.283585 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:19.284424 (+ 839us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:30:19.284791 (+ 367us) server_negotiation.cc:1035] Waiting for connection context
0810 11:30:19.285298 (+ 507us) server_negotiation.cc:299] Negotiation successful
0810 11:30:19.285898 (+ 600us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":649,"thread_start_us":428,"threads_started":1}
I20260810 11:30:19.296305 10084 raft_consensus.cc:348] T 00000000000000000000000000000000 P ed0668521a7b4c0e82440311f75dd278 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed0668521a7b4c0e82440311f75dd278" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 40395 } }
I20260810 11:30:19.296731 10084 raft_consensus.cc:374] T 00000000000000000000000000000000 P ed0668521a7b4c0e82440311f75dd278 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:30:19.296829 10084 raft_consensus.cc:729] T 00000000000000000000000000000000 P ed0668521a7b4c0e82440311f75dd278 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed0668521a7b4c0e82440311f75dd278, State: Initialized, Role: FOLLOWER
I20260810 11:30:19.297535 10084 consensus_queue.cc:260] T 00000000000000000000000000000000 P ed0668521a7b4c0e82440311f75dd278 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed0668521a7b4c0e82440311f75dd278" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 40395 } }
I20260810 11:30:19.297785 10084 raft_consensus.cc:388] T 00000000000000000000000000000000 P ed0668521a7b4c0e82440311f75dd278 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 11:30:19.297901 10084 raft_consensus.cc:482] T 00000000000000000000000000000000 P ed0668521a7b4c0e82440311f75dd278 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 11:30:19.298029 10084 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ed0668521a7b4c0e82440311f75dd278 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:30:19.300866 10084 raft_consensus.cc:504] T 00000000000000000000000000000000 P ed0668521a7b4c0e82440311f75dd278 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed0668521a7b4c0e82440311f75dd278" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 40395 } }
I20260810 11:30:19.301522 10084 leader_election.cc:302] T 00000000000000000000000000000000 P ed0668521a7b4c0e82440311f75dd278 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ed0668521a7b4c0e82440311f75dd278; no voters:
I20260810 11:30:19.302577 10084 leader_election.cc:288] T 00000000000000000000000000000000 P ed0668521a7b4c0e82440311f75dd278 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 11:30:19.302759 10089 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ed0668521a7b4c0e82440311f75dd278 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:30:19.303517 10089 raft_consensus.cc:686] T 00000000000000000000000000000000 P ed0668521a7b4c0e82440311f75dd278 [term 1 LEADER]: Becoming Leader. State: Replica: ed0668521a7b4c0e82440311f75dd278, State: Running, Role: LEADER
I20260810 11:30:19.304160 10089 consensus_queue.cc:237] T 00000000000000000000000000000000 P ed0668521a7b4c0e82440311f75dd278 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed0668521a7b4c0e82440311f75dd278" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 40395 } }
I20260810 11:30:19.304951 10084 sys_catalog.cc:564] T 00000000000000000000000000000000 P ed0668521a7b4c0e82440311f75dd278 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 11:30:19.307255 10090 sys_catalog.cc:455] T 00000000000000000000000000000000 P ed0668521a7b4c0e82440311f75dd278 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ed0668521a7b4c0e82440311f75dd278" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed0668521a7b4c0e82440311f75dd278" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 40395 } } }
I20260810 11:30:19.307672 10090 sys_catalog.cc:458] T 00000000000000000000000000000000 P ed0668521a7b4c0e82440311f75dd278 [sys.catalog]: This master's current role is: LEADER
I20260810 11:30:19.309976 10091 sys_catalog.cc:455] T 00000000000000000000000000000000 P ed0668521a7b4c0e82440311f75dd278 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ed0668521a7b4c0e82440311f75dd278. Latest consensus state: current_term: 1 leader_uuid: "ed0668521a7b4c0e82440311f75dd278" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed0668521a7b4c0e82440311f75dd278" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 40395 } } }
I20260810 11:30:19.310408 10091 sys_catalog.cc:458] T 00000000000000000000000000000000 P ed0668521a7b4c0e82440311f75dd278 [sys.catalog]: This master's current role is: LEADER
I20260810 11:30:19.311479 10096 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 11:30:19.316665 10096 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 11:30:19.327073 10096 catalog_manager.cc:1347] Generated new cluster ID: d851b1049fdf46a7a384484cae10807e
I20260810 11:30:19.327234 10096 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 11:30:19.348059 10096 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 11:30:19.349969 10096 catalog_manager.cc:1504] Loading token signing keys...
I20260810 11:30:19.364699 10096 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ed0668521a7b4c0e82440311f75dd278: Generated new TSK 0
I20260810 11:30:19.365757 10096 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.18.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.18.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.18.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.193.keytab.
Entry for principal kudu/127.8.18.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.193.keytab.
W20260810 11:30:20.462395 8267 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.8.18.193: real 1.083s user 0.000s sys 0.014s
I20260810 11:30:20.463443 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786361401922362-8267-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786361401922362-8267-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786361401922362-8267-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786361401922362-8267-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.18.193:0
--local_ip_for_outbound_sockets=127.8.18.193
--webserver_interface=127.8.18.193
--webserver_port=0
--tserver_master_addrs=127.8.18.254:40395
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.193.keytab
--principal=kudu/127.8.18.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.18.212:43379
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786361401922362-8267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786361401922362-8267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786361401922362-8267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:30:20.822438 10112 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:30:20.822785 10112 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:30:20.822903 10112 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:30:20.830745 10112 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:30:20.830895 10112 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:30:20.831058 10112 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260810 11:30:20.831130 10112 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 11:30:20.831199 10112 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.18.193
I20260810 11:30:20.841776 10112 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.18.212:43379
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786361401922362-8267-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786361401922362-8267-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.193.keytab
--principal=kudu/127.8.18.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.18.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786361401922362-8267-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.18.193
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.18.254:40395
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.18.193
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786361401922362-8267-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:30:20.843289 10112 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:30:20.844851 10112 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:30:20.859719 10117 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:30:20.859596 10120 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:30:20.859709 10118 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:30:20.860512 10112 server_base.cc:1034] running on GCE node
I20260810 11:30:20.861264 10112 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:30:20.862473 10112 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:30:20.863808 10112 hybrid_clock.cc:648] HybridClock initialized: now 1786361420863744 us; error 53 us; skew 500 ppm
I20260810 11:30:20.864250 10112 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:30:20 dist-test-slave-srgv krb5kdc[9992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361420, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:30:20.868113 10112 init.cc:377] Logged in from keytab as kudu/127.8.18.193@KRBTEST.COM (short username kudu)
I20260810 11:30:20.870100 10112 webserver.cc:460] Webserver started at http://127.8.18.193:45173/ using document root <none> and password file <none>
I20260810 11:30:20.870879 10112 fs_manager.cc:359] Metadata directory not provided
I20260810 11:30:20.871035 10112 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:30:20.871419 10112 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:30:20.874881 10112 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786361401922362-8267-0/minicluster-data/ts-0/data/instance:
uuid: "524610df19f84cfd88b4b7d4c3aaacdc"
format_stamp: "Formatted at 2026-08-10 11:30:20 on dist-test-slave-srgv"
server_key: "c12b674f70570a774ab91283dd71b49f"
server_key_iv: "e6cd6b4b24d40c7d712380e0ca71bc3c"
server_key_version: "encryptionkey@0"
I20260810 11:30:20.875851 10112 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786361401922362-8267-0/minicluster-data/ts-0/wal/instance:
uuid: "524610df19f84cfd88b4b7d4c3aaacdc"
format_stamp: "Formatted at 2026-08-10 11:30:20 on dist-test-slave-srgv"
server_key: "c12b674f70570a774ab91283dd71b49f"
server_key_iv: "e6cd6b4b24d40c7d712380e0ca71bc3c"
server_key_version: "encryptionkey@0"
I20260810 11:30:20.882345 10112 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.004s
I20260810 11:30:20.886875 10127 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:30:20.888688 10112 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260810 11:30:20.888929 10112 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786361401922362-8267-0/minicluster-data/ts-0/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786361401922362-8267-0/minicluster-data/ts-0/wal
uuid: "524610df19f84cfd88b4b7d4c3aaacdc"
format_stamp: "Formatted at 2026-08-10 11:30:20 on dist-test-slave-srgv"
server_key: "c12b674f70570a774ab91283dd71b49f"
server_key_iv: "e6cd6b4b24d40c7d712380e0ca71bc3c"
server_key_version: "encryptionkey@0"
I20260810 11:30:20.889192 10112 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786361401922362-8267-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786361401922362-8267-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786361401922362-8267-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:30:20.909624 10112 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:30:20.914510 10112 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:30:20.914860 10112 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:30:20.917966 10112 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:30:20.918202 10112 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260810 11:30:20.918334 10112 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:30:20.918439 10112 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 11:30:20 dist-test-slave-srgv krb5kdc[9992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361420, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.193@KRBTEST.COM for kudu/127.8.18.254@KRBTEST.COM
I20260810 11:30:20.949625 10136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:20.919005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.193:54317 (local address 127.8.18.254:40395)
0810 11:30:20.919602 (+ 597us) server_negotiation.cc:206] Beginning negotiation
0810 11:30:20.919609 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:30:20.921103 (+ 1494us) server_negotiation.cc:407] Connection header received
0810 11:30:20.924193 (+ 3090us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:30:20.924200 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:30:20.924332 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:30:20.924472 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:30:20.928091 (+ 3619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:20.929247 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:20.932035 (+ 2788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:20.932698 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:20.932856 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:20.940115 (+ 7259us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:30:20.940124 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:30:20.940131 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:30:20.940196 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:30:20.944814 (+ 4618us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:20.946297 (+ 1483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:20.946303 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:20.946306 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:20.946549 (+ 243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:20.947122 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:20.947128 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:20.947131 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:20.947541 (+ 410us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:30:20.947746 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0810 11:30:20.948877 (+ 1131us) server_negotiation.cc:299] Negotiation successful
0810 11:30:20.949149 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":452,"thread_start_us":318,"threads_started":1}
I20260810 11:30:20.951067 10137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:20.919799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:40395 (local address 127.8.18.193:54317)
0810 11:30:20.920833 (+ 1034us) negotiation.cc:107] Waiting for socket to connect
0810 11:30:20.920891 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0810 11:30:20.923722 (+ 2831us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:30:20.924924 (+ 1202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:30:20.924940 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:30:20.926230 (+ 1290us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:30:20.927713 (+ 1483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:30:20.927768 (+ 55us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:30:20.929585 (+ 1817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:30:20.929590 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:30:20.931787 (+ 2197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:30:20.931808 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:30:20.933020 (+ 1212us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:30:20.933030 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:30:20.933304 (+ 274us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:20.934444 (+ 1140us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:30:20.934499 (+ 55us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:30:20.939586 (+ 5087us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:30:20.945070 (+ 5484us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:30:20.945085 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:30:20.945114 (+ 29us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:30:20.946098 (+ 984us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:30:20.946727 (+ 629us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:30:20.946731 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:30:20.946734 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:30:20.946966 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:30:20.947831 (+ 865us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:30:20.947841 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:30:20.948421 (+ 580us) client_negotiation.cc:769] Sending connection context
0810 11:30:20.948769 (+ 348us) client_negotiation.cc:240] Negotiation successful
0810 11:30:20.949505 (+ 736us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":711,"thread_start_us":466,"threads_started":1}
I20260810 11:30:20.963970 10112 rpc_server.cc:307] RPC server started. Bound to: 127.8.18.193:35655
I20260810 11:30:20.964046 10249 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.18.193:35655 every 8 connection(s)
I20260810 11:30:20.965629 10112 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786361401922362-8267-0/minicluster-data/ts-0/data/info.pb
I20260810 11:30:20.969029 10252 heartbeater.cc:347] Connected to a master server at 127.8.18.254:40395
I20260810 11:30:20.969089 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 10112
I20260810 11:30:20.969340 8267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786361401922362-8267-0/minicluster-data/ts-0/wal/instance
I20260810 11:30:20.969377 10252 heartbeater.cc:464] Registering TS with master...
I20260810 11:30:20.970270 8267 external_mini_cluster.cc:1428] Setting key eb014d655a7d205d609338a9f75b9eb5
I20260810 11:30:20.970496 10252 heartbeater.cc:511] Master 127.8.18.254:40395 requested a full tablet report, sending...
I20260810 11:30:20.973381 10037 ts_manager.cc:194] Registered new tserver with Master: 524610df19f84cfd88b4b7d4c3aaacdc (127.8.18.193:35655)
I20260810 11:30:20.975294 10037 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.18.193@KRBTEST.COM'} at 127.8.18.193:54317
I20260810 11:30:20.978123 10136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:20.961113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.193:44625 (local address 127.8.18.254:40395)
0810 11:30:20.961322 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0810 11:30:20.961329 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:30:20.962302 (+ 973us) server_negotiation.cc:407] Connection header received
0810 11:30:20.962583 (+ 281us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:30:20.962590 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:30:20.962691 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:30:20.962810 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:30:20.964337 (+ 1527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:20.965097 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:20.967052 (+ 1955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:20.967774 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:20.967900 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:20.970486 (+ 2586us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:30:20.970492 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:30:20.970495 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:30:20.970539 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:30:20.974392 (+ 3853us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:20.975524 (+ 1132us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:20.975530 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:20.975534 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:20.975730 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:20.976253 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:20.976262 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:20.976268 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:20.976723 (+ 455us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:30:20.976900 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0810 11:30:20.977479 (+ 579us) server_negotiation.cc:299] Negotiation successful
0810 11:30:20.977733 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260810 11:30:20.978425 10245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:20.961112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:40395 (local address 127.8.18.193:44625)
0810 11:30:20.962150 (+ 1038us) negotiation.cc:107] Waiting for socket to connect
0810 11:30:20.962173 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0810 11:30:20.962394 (+ 221us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:30:20.963088 (+ 694us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:30:20.963093 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:30:20.963641 (+ 548us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:30:20.964061 (+ 420us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:30:20.964075 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:30:20.965302 (+ 1227us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:30:20.965307 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:30:20.966797 (+ 1490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:30:20.966830 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:30:20.968024 (+ 1194us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:30:20.968032 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:30:20.968241 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:20.969127 (+ 886us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:30:20.969187 (+ 60us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:30:20.970247 (+ 1060us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:30:20.974610 (+ 4363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:30:20.974619 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:30:20.974627 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:30:20.975338 (+ 711us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:30:20.975889 (+ 551us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:30:20.975897 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:30:20.975904 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:30:20.976091 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:30:20.976871 (+ 780us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:30:20.976877 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:30:20.977141 (+ 264us) client_negotiation.cc:769] Sending connection context
0810 11:30:20.977309 (+ 168us) client_negotiation.cc:240] Negotiation successful
0810 11:30:20.977733 (+ 424us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":850,"thread_start_us":503,"threads_started":1}
WARNING: no policy specified for kudu/127.8.18.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.18.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.18.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.194.keytab.
Entry for principal kudu/127.8.18.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.194.keytab.
I20260810 11:30:21.021636 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786361401922362-8267-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786361401922362-8267-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786361401922362-8267-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786361401922362-8267-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.18.194:0
--local_ip_for_outbound_sockets=127.8.18.194
--webserver_interface=127.8.18.194
--webserver_port=0
--tserver_master_addrs=127.8.18.254:40395
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.194.keytab
--principal=kudu/127.8.18.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.18.212:43379
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786361401922362-8267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786361401922362-8267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786361401922362-8267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:30:21.379016 10256 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:30:21.379426 10256 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:30:21.379622 10256 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:30:21.387396 10256 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:30:21.387594 10256 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:30:21.387758 10256 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260810 11:30:21.387852 10256 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 11:30:21.387998 10256 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.18.194
I20260810 11:30:21.398890 10256 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.18.212:43379
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786361401922362-8267-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786361401922362-8267-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.194.keytab
--principal=kudu/127.8.18.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.18.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786361401922362-8267-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.18.194
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.18.254:40395
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.18.194
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786361401922362-8267-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:30:21.400367 10256 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:30:21.401883 10256 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:30:21.416640 10261 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:30:21.416575 10264 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:30:21.416591 10262 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:30:21.417452 10256 server_base.cc:1034] running on GCE node
I20260810 11:30:21.418182 10256 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:30:21.419270 10256 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:30:21.420493 10256 hybrid_clock.cc:648] HybridClock initialized: now 1786361421420420 us; error 68 us; skew 500 ppm
I20260810 11:30:21.420862 10256 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:30:21 dist-test-slave-srgv krb5kdc[9992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361421, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:30:21.424525 10256 init.cc:377] Logged in from keytab as kudu/127.8.18.194@KRBTEST.COM (short username kudu)
I20260810 11:30:21.426416 10256 webserver.cc:460] Webserver started at http://127.8.18.194:39317/ using document root <none> and password file <none>
I20260810 11:30:21.427172 10256 fs_manager.cc:359] Metadata directory not provided
I20260810 11:30:21.427356 10256 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:30:21.427712 10256 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:30:21.430970 10256 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786361401922362-8267-0/minicluster-data/ts-1/data/instance:
uuid: "600d04a3960143f89a8e1bee5bcd65e7"
format_stamp: "Formatted at 2026-08-10 11:30:21 on dist-test-slave-srgv"
server_key: "24580fb9e22cdc9f79e872897c3fefbe"
server_key_iv: "520ee3f0ee1ba0821fe93c38e7b8a835"
server_key_version: "encryptionkey@0"
I20260810 11:30:21.431838 10256 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786361401922362-8267-0/minicluster-data/ts-1/wal/instance:
uuid: "600d04a3960143f89a8e1bee5bcd65e7"
format_stamp: "Formatted at 2026-08-10 11:30:21 on dist-test-slave-srgv"
server_key: "24580fb9e22cdc9f79e872897c3fefbe"
server_key_iv: "520ee3f0ee1ba0821fe93c38e7b8a835"
server_key_version: "encryptionkey@0"
I20260810 11:30:21.437999 10256 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260810 11:30:21.442035 10271 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:30:21.443539 10256 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260810 11:30:21.443751 10256 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786361401922362-8267-0/minicluster-data/ts-1/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786361401922362-8267-0/minicluster-data/ts-1/wal
uuid: "600d04a3960143f89a8e1bee5bcd65e7"
format_stamp: "Formatted at 2026-08-10 11:30:21 on dist-test-slave-srgv"
server_key: "24580fb9e22cdc9f79e872897c3fefbe"
server_key_iv: "520ee3f0ee1ba0821fe93c38e7b8a835"
server_key_version: "encryptionkey@0"
I20260810 11:30:21.443992 10256 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786361401922362-8267-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786361401922362-8267-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786361401922362-8267-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:30:21.462709 10256 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:30:21.467833 10256 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:30:21.468199 10256 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:30:21.471470 10256 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:30:21.471616 10256 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:30:21.471772 10256 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:30:21.471863 10256 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 11:30:21 dist-test-slave-srgv krb5kdc[9992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361421, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.194@KRBTEST.COM for kudu/127.8.18.254@KRBTEST.COM
I20260810 11:30:21.497298 10136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:21.471970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.194:38691 (local address 127.8.18.254:40395)
0810 11:30:21.472237 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0810 11:30:21.472246 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 11:30:21.473952 (+ 1706us) server_negotiation.cc:407] Connection header received
0810 11:30:21.476441 (+ 2489us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:30:21.476447 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:30:21.476569 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:30:21.476699 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:30:21.479497 (+ 2798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:21.480607 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:21.483760 (+ 3153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:21.484413 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:21.484588 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:21.489954 (+ 5366us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:30:21.489963 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:30:21.489968 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:30:21.490023 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:30:21.493718 (+ 3695us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:21.494739 (+ 1021us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:21.494745 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:21.494749 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:21.494902 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:21.495403 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:21.495409 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:21.495413 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:21.495735 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:30:21.495899 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0810 11:30:21.496660 (+ 761us) server_negotiation.cc:299] Negotiation successful
0810 11:30:21.496910 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260810 11:30:21.498956 10280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:21.472746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:40395 (local address 127.8.18.194:38691)
0810 11:30:21.473755 (+ 1009us) negotiation.cc:107] Waiting for socket to connect
0810 11:30:21.473805 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0810 11:30:21.476075 (+ 2270us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:30:21.477062 (+ 987us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:30:21.477087 (+ 25us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:30:21.478112 (+ 1025us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:30:21.479254 (+ 1142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:30:21.479279 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:30:21.480841 (+ 1562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:30:21.480845 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:30:21.483509 (+ 2664us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:30:21.483525 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:30:21.484707 (+ 1182us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:30:21.484716 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:30:21.484977 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:21.485891 (+ 914us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:30:21.485938 (+ 47us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:30:21.489550 (+ 3612us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:30:21.493970 (+ 4420us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:30:21.493979 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:30:21.493994 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:30:21.494581 (+ 587us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:30:21.495027 (+ 446us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:30:21.495033 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:30:21.495036 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:30:21.495256 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:30:21.495872 (+ 616us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:30:21.495881 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:30:21.496275 (+ 394us) client_negotiation.cc:769] Sending connection context
0810 11:30:21.496560 (+ 285us) client_negotiation.cc:240] Negotiation successful
0810 11:30:21.497149 (+ 589us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":708,"thread_start_us":402,"threads_started":1}
I20260810 11:30:21.526225 10136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:21.508605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.194:47851 (local address 127.8.18.254:40395)
0810 11:30:21.508874 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0810 11:30:21.508884 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 11:30:21.510111 (+ 1227us) server_negotiation.cc:407] Connection header received
0810 11:30:21.510357 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:30:21.510365 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:30:21.510470 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:30:21.510659 (+ 189us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:30:21.512218 (+ 1559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:21.513376 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:21.514950 (+ 1574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:21.515412 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:21.515591 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:21.518399 (+ 2808us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:30:21.518405 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:30:21.518408 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:30:21.518448 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:30:21.522349 (+ 3901us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:21.523450 (+ 1101us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:21.523455 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:21.523457 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:21.523608 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:21.524256 (+ 648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:21.524264 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:21.524268 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:21.524795 (+ 527us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:30:21.524987 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0810 11:30:21.525663 (+ 676us) server_negotiation.cc:299] Negotiation successful
0810 11:30:21.525963 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260810 11:30:21.526700 10339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:21.508548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:40395 (local address 127.8.18.194:47851)
0810 11:30:21.509921 (+ 1373us) negotiation.cc:107] Waiting for socket to connect
0810 11:30:21.509951 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0810 11:30:21.510169 (+ 218us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:30:21.510683 (+ 514us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:30:21.510688 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:30:21.511677 (+ 989us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:30:21.512038 (+ 361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:30:21.512049 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:30:21.513628 (+ 1579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:30:21.513638 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:30:21.514785 (+ 1147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:30:21.514801 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:30:21.515665 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:30:21.515673 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:30:21.515860 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:21.516690 (+ 830us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:30:21.516709 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:30:21.518167 (+ 1458us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:30:21.522571 (+ 4404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:30:21.522577 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:30:21.522581 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:30:21.523287 (+ 706us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:30:21.523758 (+ 471us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:30:21.523764 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:30:21.523767 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:30:21.523980 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:30:21.524959 (+ 979us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:30:21.524971 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:30:21.525263 (+ 292us) client_negotiation.cc:769] Sending connection context
0810 11:30:21.525502 (+ 239us) client_negotiation.cc:240] Negotiation successful
0810 11:30:21.526044 (+ 542us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1128,"thread_start_us":806,"threads_started":1}
I20260810 11:30:21.536561 10256 rpc_server.cc:307] RPC server started. Bound to: 127.8.18.194:45797
I20260810 11:30:21.536590 10391 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.18.194:45797 every 8 connection(s)
I20260810 11:30:21.538012 10256 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786361401922362-8267-0/minicluster-data/ts-1/data/info.pb
I20260810 11:30:21.540432 10392 heartbeater.cc:347] Connected to a master server at 127.8.18.254:40395
I20260810 11:30:21.540721 10392 heartbeater.cc:464] Registering TS with master...
I20260810 11:30:21.541477 10392 heartbeater.cc:511] Master 127.8.18.254:40395 requested a full tablet report, sending...
I20260810 11:30:21.542943 10037 ts_manager.cc:194] Registered new tserver with Master: 600d04a3960143f89a8e1bee5bcd65e7 (127.8.18.194:45797)
I20260810 11:30:21.543769 10037 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.18.194@KRBTEST.COM'} at 127.8.18.194:38691
I20260810 11:30:21.545284 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 10256
I20260810 11:30:21.545501 8267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786361401922362-8267-0/minicluster-data/ts-1/wal/instance
I20260810 11:30:21.545981 8267 external_mini_cluster.cc:1428] Setting key 0e722593c806f6b553c258a35615c594
WARNING: no policy specified for kudu/127.8.18.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.18.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.18.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.195.keytab.
Entry for principal kudu/127.8.18.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.195.keytab.
I20260810 11:30:21.593856 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786361401922362-8267-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786361401922362-8267-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786361401922362-8267-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786361401922362-8267-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.18.195:0
--local_ip_for_outbound_sockets=127.8.18.195
--webserver_interface=127.8.18.195
--webserver_port=0
--tserver_master_addrs=127.8.18.254:40395
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.195.keytab
--principal=kudu/127.8.18.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.18.212:43379
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786361401922362-8267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786361401922362-8267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786361401922362-8267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:30:21.955988 10398 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:30:21.956321 10398 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:30:21.956506 10398 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:30:21.965309 10398 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:30:21.965529 10398 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:30:21.965685 10398 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260810 11:30:21.965752 10398 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 11:30:21.965890 10398 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.18.195
I20260810 11:30:21.976773 10398 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.18.212:43379
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786361401922362-8267-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786361401922362-8267-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.195.keytab
--principal=kudu/127.8.18.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.18.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786361401922362-8267-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.18.195
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.18.254:40395
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.18.195
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786361401922362-8267-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:30:21.978428 10398 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:30:21.979993 10398 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 11:30:21.979995 10252 heartbeater.cc:503] Master 127.8.18.254:40395 was elected leader, sending a full tablet report...
W20260810 11:30:21.995704 10406 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:30:21.995573 10403 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:30:21.995535 10404 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:30:21.996348 10398 server_base.cc:1034] running on GCE node
I20260810 11:30:21.997067 10398 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:30:21.998636 10398 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:30:21.999933 10398 hybrid_clock.cc:648] HybridClock initialized: now 1786361421999851 us; error 65 us; skew 500 ppm
I20260810 11:30:22.000386 10398 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:30:22 dist-test-slave-srgv krb5kdc[9992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361422, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:30:22.004313 10398 init.cc:377] Logged in from keytab as kudu/127.8.18.195@KRBTEST.COM (short username kudu)
I20260810 11:30:22.006175 10398 webserver.cc:460] Webserver started at http://127.8.18.195:37191/ using document root <none> and password file <none>
I20260810 11:30:22.007064 10398 fs_manager.cc:359] Metadata directory not provided
I20260810 11:30:22.007237 10398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:30:22.007603 10398 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:30:22.011093 10398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786361401922362-8267-0/minicluster-data/ts-2/data/instance:
uuid: "2972efc435eb470c88b7c8689c4b96e1"
format_stamp: "Formatted at 2026-08-10 11:30:22 on dist-test-slave-srgv"
server_key: "fce52b22ed16e5cd4fa70aed2d13ea6b"
server_key_iv: "0e7957549871cd8cb109f6f4fa2b0419"
server_key_version: "encryptionkey@0"
I20260810 11:30:22.012351 10398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786361401922362-8267-0/minicluster-data/ts-2/wal/instance:
uuid: "2972efc435eb470c88b7c8689c4b96e1"
format_stamp: "Formatted at 2026-08-10 11:30:22 on dist-test-slave-srgv"
server_key: "fce52b22ed16e5cd4fa70aed2d13ea6b"
server_key_iv: "0e7957549871cd8cb109f6f4fa2b0419"
server_key_version: "encryptionkey@0"
I20260810 11:30:22.019024 10398 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260810 11:30:22.023267 10413 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:30:22.024928 10398 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 11:30:22.025175 10398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786361401922362-8267-0/minicluster-data/ts-2/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786361401922362-8267-0/minicluster-data/ts-2/wal
uuid: "2972efc435eb470c88b7c8689c4b96e1"
format_stamp: "Formatted at 2026-08-10 11:30:22 on dist-test-slave-srgv"
server_key: "fce52b22ed16e5cd4fa70aed2d13ea6b"
server_key_iv: "0e7957549871cd8cb109f6f4fa2b0419"
server_key_version: "encryptionkey@0"
I20260810 11:30:22.025486 10398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786361401922362-8267-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786361401922362-8267-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786361401922362-8267-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:30:22.042088 10398 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:30:22.047096 10398 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:30:22.047508 10398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:30:22.050774 10398 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:30:22.050992 10398 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:30:22.051155 10398 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:30:22.051263 10398 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 11:30:22 dist-test-slave-srgv krb5kdc[9992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361422, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.195@KRBTEST.COM for kudu/127.8.18.254@KRBTEST.COM
I20260810 11:30:22.082598 10422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:22.051463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.195:36301 (local address 127.8.18.254:40395)
0810 11:30:22.052355 (+ 892us) server_negotiation.cc:206] Beginning negotiation
0810 11:30:22.052361 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:30:22.053472 (+ 1111us) server_negotiation.cc:407] Connection header received
0810 11:30:22.056329 (+ 2857us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:30:22.056336 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:30:22.056451 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:30:22.056609 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:30:22.060598 (+ 3989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:22.061756 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:22.066013 (+ 4257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:22.066489 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:22.066631 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:22.073644 (+ 7013us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:30:22.073655 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:30:22.073663 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:30:22.073733 (+ 70us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:30:22.078311 (+ 4578us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:22.079776 (+ 1465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:22.079782 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:22.079784 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:22.079944 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:22.080484 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:22.080489 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:22.080491 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:22.080804 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:30:22.081001 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0810 11:30:22.081915 (+ 914us) server_negotiation.cc:299] Negotiation successful
0810 11:30:22.082185 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":738,"thread_start_us":592,"threads_started":1}
I20260810 11:30:22.083783 10423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:22.052244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:40395 (local address 127.8.18.195:36301)
0810 11:30:22.053225 (+ 981us) negotiation.cc:107] Waiting for socket to connect
0810 11:30:22.053276 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0810 11:30:22.055865 (+ 2589us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:30:22.057227 (+ 1362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:30:22.057248 (+ 21us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:30:22.058708 (+ 1460us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:30:22.060164 (+ 1456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:30:22.060212 (+ 48us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:30:22.062046 (+ 1834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:30:22.062064 (+ 18us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:30:22.065678 (+ 3614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:30:22.065725 (+ 47us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:30:22.066844 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:30:22.066856 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:30:22.067319 (+ 463us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:22.068332 (+ 1013us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:30:22.068378 (+ 46us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:30:22.073169 (+ 4791us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:30:22.078651 (+ 5482us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:30:22.078665 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:30:22.078701 (+ 36us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:30:22.079558 (+ 857us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:30:22.080083 (+ 525us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:30:22.080088 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:30:22.080092 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:30:22.080334 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:30:22.080958 (+ 624us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:30:22.080968 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:30:22.081393 (+ 425us) client_negotiation.cc:769] Sending connection context
0810 11:30:22.081770 (+ 377us) client_negotiation.cc:240] Negotiation successful
0810 11:30:22.082387 (+ 617us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":682,"thread_start_us":436,"threads_started":1}
I20260810 11:30:22.108886 10422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:22.091892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.195:49825 (local address 127.8.18.254:40395)
0810 11:30:22.092179 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0810 11:30:22.092189 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 11:30:22.092847 (+ 658us) server_negotiation.cc:407] Connection header received
0810 11:30:22.093072 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:30:22.093082 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:30:22.093200 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:30:22.093328 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:30:22.095071 (+ 1743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:22.096033 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:22.097779 (+ 1746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:22.098405 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:22.098564 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:22.101572 (+ 3008us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:30:22.101578 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:30:22.101581 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:30:22.101619 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:30:22.105243 (+ 3624us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:22.106487 (+ 1244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:22.106492 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:22.106495 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:22.106638 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:22.107139 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:22.107143 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:22.107144 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:22.107442 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:30:22.107573 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0810 11:30:22.108141 (+ 568us) server_negotiation.cc:299] Negotiation successful
0810 11:30:22.108418 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260810 11:30:22.109803 10497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:22.091854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:40395 (local address 127.8.18.195:49825)
0810 11:30:22.092681 (+ 827us) negotiation.cc:107] Waiting for socket to connect
0810 11:30:22.092711 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0810 11:30:22.092913 (+ 202us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:30:22.093390 (+ 477us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:30:22.093397 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:30:22.094085 (+ 688us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:30:22.094771 (+ 686us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:30:22.094856 (+ 85us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:30:22.096316 (+ 1460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:30:22.096328 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:30:22.097600 (+ 1272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:30:22.097615 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:30:22.098657 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:30:22.098664 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:30:22.098842 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:22.099658 (+ 816us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:30:22.099688 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:30:22.101255 (+ 1567us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:30:22.105457 (+ 4202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:30:22.105466 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:30:22.105474 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:30:22.106327 (+ 853us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:30:22.106771 (+ 444us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:30:22.106779 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:30:22.106786 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:30:22.106996 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:30:22.107563 (+ 567us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:30:22.107571 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:30:22.107860 (+ 289us) client_negotiation.cc:769] Sending connection context
0810 11:30:22.108046 (+ 186us) client_negotiation.cc:240] Negotiation successful
0810 11:30:22.108583 (+ 537us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":620,"thread_start_us":435,"threads_started":1}
I20260810 11:30:22.115459 10398 rpc_server.cc:307] RPC server started. Bound to: 127.8.18.195:38593
I20260810 11:30:22.115491 10534 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.18.195:38593 every 8 connection(s)
I20260810 11:30:22.116873 10398 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786361401922362-8267-0/minicluster-data/ts-2/data/info.pb
I20260810 11:30:22.119120 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 10398
I20260810 11:30:22.119284 10535 heartbeater.cc:347] Connected to a master server at 127.8.18.254:40395
I20260810 11:30:22.119391 8267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786361401922362-8267-0/minicluster-data/ts-2/wal/instance
I20260810 11:30:22.119509 10535 heartbeater.cc:464] Registering TS with master...
I20260810 11:30:22.120102 8267 external_mini_cluster.cc:1428] Setting key d6cf0108c73ccfe7658d20c70739c041
I20260810 11:30:22.120244 10535 heartbeater.cc:511] Master 127.8.18.254:40395 requested a full tablet report, sending...
I20260810 11:30:22.122061 10038 ts_manager.cc:194] Registered new tserver with Master: 2972efc435eb470c88b7c8689c4b96e1 (127.8.18.195:38593)
I20260810 11:30:22.123805 8267 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 11:30:22.123800 10038 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.18.195@KRBTEST.COM'} at 127.8.18.195:36301
Aug 10 11:30:22 dist-test-slave-srgv krb5kdc[9992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361422, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 10 11:30:22 dist-test-slave-srgv krb5kdc[9992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361422, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.18.254@KRBTEST.COM
I20260810 11:30:22.181752 10422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:22.166751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41810 (local address 127.8.18.254:40395)
0810 11:30:22.167032 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0810 11:30:22.167039 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:30:22.167286 (+ 247us) server_negotiation.cc:407] Connection header received
0810 11:30:22.167474 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:30:22.167478 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:30:22.167552 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:30:22.167648 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:30:22.169180 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:22.170051 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:22.171329 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:22.171745 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:22.171846 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:22.176145 (+ 4299us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:30:22.176151 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:30:22.176154 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:30:22.176194 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:30:22.178886 (+ 2692us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:22.179610 (+ 724us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:22.179619 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:22.179622 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:22.179765 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:22.180165 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:22.180170 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:22.180173 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:22.180484 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:30:22.180756 (+ 272us) server_negotiation.cc:1035] Waiting for connection context
0810 11:30:22.181063 (+ 307us) server_negotiation.cc:299] Negotiation successful
0810 11:30:22.181318 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
I20260810 11:30:22.186074 10038 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41810:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 11:30:22.193197 10038 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 11:30:22.221832 10550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:22.207270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40806 (local address 127.8.18.195:38593)
0810 11:30:22.210323 (+ 3053us) server_negotiation.cc:206] Beginning negotiation
0810 11:30:22.210329 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:30:22.210358 (+ 29us) server_negotiation.cc:407] Connection header received
0810 11:30:22.210509 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:30:22.210517 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:30:22.210785 (+ 268us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:30:22.211030 (+ 245us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:30:22.212326 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:22.214626 (+ 2300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:22.218486 (+ 3860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:22.220104 (+ 1618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:22.220247 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:22.220525 (+ 278us) server_negotiation.cc:1035] Waiting for connection context
0810 11:30:22.221255 (+ 730us) server_negotiation.cc:299] Negotiation successful
0810 11:30:22.221474 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2911,"thread_start_us":2726,"threads_started":1}
I20260810 11:30:22.221975 10553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:22.207477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.193:35655 (local address 127.0.0.1:47350)
0810 11:30:22.212947 (+ 5470us) negotiation.cc:107] Waiting for socket to connect
0810 11:30:22.212968 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 11:30:22.213119 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:30:22.214039 (+ 920us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:30:22.214044 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:30:22.214075 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:30:22.214639 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:30:22.214652 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:30:22.216671 (+ 2019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:30:22.216676 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:30:22.219173 (+ 2497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:30:22.219188 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:30:22.221096 (+ 1908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:30:22.221102 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:30:22.221274 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:22.221302 (+ 28us) client_negotiation.cc:769] Sending connection context
0810 11:30:22.221391 (+ 89us) client_negotiation.cc:240] Negotiation successful
0810 11:30:22.221543 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5309,"mutex_wait_us":200,"thread_start_us":5828,"threads_started":1}
I20260810 11:30:22.222131 10549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:22.207129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.195:38593 (local address 127.0.0.1:40806)
0810 11:30:22.208040 (+ 911us) negotiation.cc:107] Waiting for socket to connect
0810 11:30:22.208088 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0810 11:30:22.208289 (+ 201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:30:22.211396 (+ 3107us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:30:22.211404 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:30:22.211435 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:30:22.212043 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:30:22.212061 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:30:22.214804 (+ 2743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:30:22.214811 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:30:22.218236 (+ 3425us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:30:22.218252 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:30:22.220689 (+ 2437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:30:22.220695 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:30:22.220861 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:22.220894 (+ 33us) client_negotiation.cc:769] Sending connection context
0810 11:30:22.221063 (+ 169us) client_negotiation.cc:240] Negotiation successful
0810 11:30:22.221710 (+ 647us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":623,"mutex_wait_us":563,"thread_start_us":1255,"threads_started":1}
I20260810 11:30:22.222414 10552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:22.208235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47350 (local address 127.8.18.193:35655)
0810 11:30:22.209737 (+ 1502us) server_negotiation.cc:206] Beginning negotiation
0810 11:30:22.209750 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 11:30:22.213085 (+ 3335us) server_negotiation.cc:407] Connection header received
0810 11:30:22.213283 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:30:22.213295 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:30:22.213595 (+ 300us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:30:22.213811 (+ 216us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:30:22.214887 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:22.216508 (+ 1621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:22.219347 (+ 2839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:22.220865 (+ 1518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:22.221003 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:22.221203 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0810 11:30:22.221501 (+ 298us) server_negotiation.cc:299] Negotiation successful
0810 11:30:22.221681 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1263,"thread_start_us":1054,"threads_started":1}
I20260810 11:30:22.222805 10551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:22.212287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.194:45797 (local address 127.0.0.1:47416)
0810 11:30:22.213491 (+ 1204us) negotiation.cc:107] Waiting for socket to connect
0810 11:30:22.213510 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 11:30:22.213626 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:30:22.214258 (+ 632us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:30:22.214263 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:30:22.214294 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:30:22.214907 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:30:22.214925 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:30:22.216726 (+ 1801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:30:22.216734 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:30:22.218665 (+ 1931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:30:22.218678 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:30:22.220748 (+ 2070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:30:22.220755 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:30:22.220955 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:22.220993 (+ 38us) client_negotiation.cc:769] Sending connection context
0810 11:30:22.221084 (+ 91us) client_negotiation.cc:240] Negotiation successful
0810 11:30:22.221317 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1030,"mutex_wait_us":87,"thread_start_us":342,"threads_started":1}
I20260810 11:30:22.222884 10554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:22.212515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47416 (local address 127.8.18.194:45797)
0810 11:30:22.213637 (+ 1122us) server_negotiation.cc:206] Beginning negotiation
0810 11:30:22.213647 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 11:30:22.213703 (+ 56us) server_negotiation.cc:407] Connection header received
0810 11:30:22.213810 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:30:22.213817 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:30:22.214029 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:30:22.214250 (+ 221us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:30:22.215205 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:22.216598 (+ 1393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:22.218840 (+ 2242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:22.220222 (+ 1382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:22.220366 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:22.220569 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0810 11:30:22.222393 (+ 1824us) server_negotiation.cc:299] Negotiation successful
0810 11:30:22.222582 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":960,"thread_start_us":999,"threads_started":1}
I20260810 11:30:22.226295 10464 tablet_service.cc:1467] Processing CreateTablet for tablet 83e28eea13a14c64b9d3a4ed5a6b7ed2 (DEFAULT_TABLE table=test-table [id=f01dd88603ed4c97ac655489cded0178]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:30:22.226330 10179 tablet_service.cc:1467] Processing CreateTablet for tablet 83e28eea13a14c64b9d3a4ed5a6b7ed2 (DEFAULT_TABLE table=test-table [id=f01dd88603ed4c97ac655489cded0178]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:30:22.226647 10321 tablet_service.cc:1467] Processing CreateTablet for tablet 83e28eea13a14c64b9d3a4ed5a6b7ed2 (DEFAULT_TABLE table=test-table [id=f01dd88603ed4c97ac655489cded0178]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:30:22.227618 10464 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 83e28eea13a14c64b9d3a4ed5a6b7ed2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:30:22.227761 10179 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 83e28eea13a14c64b9d3a4ed5a6b7ed2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:30:22.227787 10321 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 83e28eea13a14c64b9d3a4ed5a6b7ed2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:30:22.240489 10555 tablet_bootstrap.cc:492] T 83e28eea13a14c64b9d3a4ed5a6b7ed2 P 2972efc435eb470c88b7c8689c4b96e1: Bootstrap starting.
I20260810 11:30:22.243040 10556 tablet_bootstrap.cc:492] T 83e28eea13a14c64b9d3a4ed5a6b7ed2 P 524610df19f84cfd88b4b7d4c3aaacdc: Bootstrap starting.
I20260810 11:30:22.245141 10555 tablet_bootstrap.cc:654] T 83e28eea13a14c64b9d3a4ed5a6b7ed2 P 2972efc435eb470c88b7c8689c4b96e1: Neither blocks nor log segments found. Creating new log.
I20260810 11:30:22.246711 10555 log.cc:824] T 83e28eea13a14c64b9d3a4ed5a6b7ed2 P 2972efc435eb470c88b7c8689c4b96e1: Log is configured to *not* fsync() on all Append() calls
I20260810 11:30:22.247316 10556 tablet_bootstrap.cc:654] T 83e28eea13a14c64b9d3a4ed5a6b7ed2 P 524610df19f84cfd88b4b7d4c3aaacdc: Neither blocks nor log segments found. Creating new log.
I20260810 11:30:22.247880 10557 tablet_bootstrap.cc:492] T 83e28eea13a14c64b9d3a4ed5a6b7ed2 P 600d04a3960143f89a8e1bee5bcd65e7: Bootstrap starting.
I20260810 11:30:22.248734 10556 log.cc:824] T 83e28eea13a14c64b9d3a4ed5a6b7ed2 P 524610df19f84cfd88b4b7d4c3aaacdc: Log is configured to *not* fsync() on all Append() calls
I20260810 11:30:22.251950 10556 tablet_bootstrap.cc:492] T 83e28eea13a14c64b9d3a4ed5a6b7ed2 P 524610df19f84cfd88b4b7d4c3aaacdc: No bootstrap required, opened a new log
I20260810 11:30:22.252197 10555 tablet_bootstrap.cc:492] T 83e28eea13a14c64b9d3a4ed5a6b7ed2 P 2972efc435eb470c88b7c8689c4b96e1: No bootstrap required, opened a new log
I20260810 11:30:22.252262 10556 ts_tablet_manager.cc:1397] T 83e28eea13a14c64b9d3a4ed5a6b7ed2 P 524610df19f84cfd88b4b7d4c3aaacdc: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.003s
I20260810 11:30:22.252260 10557 tablet_bootstrap.cc:654] T 83e28eea13a14c64b9d3a4ed5a6b7ed2 P 600d04a3960143f89a8e1bee5bcd65e7: Neither blocks nor log segments found. Creating new log.
I20260810 11:30:22.252514 10555 ts_tablet_manager.cc:1397] T 83e28eea13a14c64b9d3a4ed5a6b7ed2 P 2972efc435eb470c88b7c8689c4b96e1: Time spent bootstrapping tablet: real 0.012s user 0.007s sys 0.000s
I20260810 11:30:22.254043 10557 log.cc:824] T 83e28eea13a14c64b9d3a4ed5a6b7ed2 P 600d04a3960143f89a8e1bee5bcd65e7: Log is configured to *not* fsync() on all Append() calls
I20260810 11:30:22.263602 10556 raft_consensus.cc:348] T 83e28eea13a14c64b9d3a4ed5a6b7ed2 P 524610df19f84cfd88b4b7d4c3aaacdc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2972efc435eb470c88b7c8689c4b96e1" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 38593 } } peers { permanent_uuid: "524610df19f84cfd88b4b7d4c3aaacdc" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 35655 } } peers { permanent_uuid: "600d04a3960143f89a8e1bee5bcd65e7" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 45797 } }
I20260810 11:30:22.264192 10556 raft_consensus.cc:374] T 83e28eea13a14c64b9d3a4ed5a6b7ed2 P 524610df19f84cfd88b4b7d4c3aaacdc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:30:22.264294 10556 raft_consensus.cc:729] T 83e28eea13a14c64b9d3a4ed5a6b7ed2 P 524610df19f84cfd88b4b7d4c3aaacdc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 524610df19f84cfd88b4b7d4c3aaacdc, State: Initialized, Role: FOLLOWER
I20260810 11:30:22.265161 10556 consensus_queue.cc:260] T 83e28eea13a14c64b9d3a4ed5a6b7ed2 P 524610df19f84cfd88b4b7d4c3aaacdc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2972efc435eb470c88b7c8689c4b96e1" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 38593 } } peers { permanent_uuid: "524610df19f84cfd88b4b7d4c3aaacdc" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 35655 } } peers { permanent_uuid: "600d04a3960143f89a8e1bee5bcd65e7" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 45797 } }
I20260810 11:30:22.267885 10555 raft_consensus.cc:348] T 83e28eea13a14c64b9d3a4ed5a6b7ed2 P 2972efc435eb470c88b7c8689c4b96e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2972efc435eb470c88b7c8689c4b96e1" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 38593 } } peers { permanent_uuid: "524610df19f84cfd88b4b7d4c3aaacdc" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 35655 } } peers { permanent_uuid: "600d04a3960143f89a8e1bee5bcd65e7" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 45797 } }
I20260810 11:30:22.268550 10555 raft_consensus.cc:374] T 83e28eea13a14c64b9d3a4ed5a6b7ed2 P 2972efc435eb470c88b7c8689c4b96e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:30:22.268688 10555 raft_consensus.cc:729] T 83e28eea13a14c64b9d3a4ed5a6b7ed2 P 2972efc435eb470c88b7c8689c4b96e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2972efc435eb470c88b7c8689c4b96e1, State: Initialized, Role: FOLLOWER
I20260810 11:30:22.269635 10555 consensus_queue.cc:260] T 83e28eea13a14c64b9d3a4ed5a6b7ed2 P 2972efc435eb470c88b7c8689c4b96e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2972efc435eb470c88b7c8689c4b96e1" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 38593 } } peers { permanent_uuid: "524610df19f84cfd88b4b7d4c3aaacdc" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 35655 } } peers { permanent_uuid: "600d04a3960143f89a8e1bee5bcd65e7" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 45797 } }
I20260810 11:30:22.273880 10556 ts_tablet_manager.cc:1428] T 83e28eea13a14c64b9d3a4ed5a6b7ed2 P 524610df19f84cfd88b4b7d4c3aaacdc: Time spent starting tablet: real 0.021s user 0.017s sys 0.005s
I20260810 11:30:22.281834 10535 heartbeater.cc:503] Master 127.8.18.254:40395 was elected leader, sending a full tablet report...
I20260810 11:30:22.282755 10555 ts_tablet_manager.cc:1428] T 83e28eea13a14c64b9d3a4ed5a6b7ed2 P 2972efc435eb470c88b7c8689c4b96e1: Time spent starting tablet: real 0.030s user 0.027s sys 0.004s
I20260810 11:30:22.283941 10557 tablet_bootstrap.cc:492] T 83e28eea13a14c64b9d3a4ed5a6b7ed2 P 600d04a3960143f89a8e1bee5bcd65e7: No bootstrap required, opened a new log
I20260810 11:30:22.284322 10557 ts_tablet_manager.cc:1397] T 83e28eea13a14c64b9d3a4ed5a6b7ed2 P 600d04a3960143f89a8e1bee5bcd65e7: Time spent bootstrapping tablet: real 0.037s user 0.016s sys 0.016s
I20260810 11:30:22.293012 10557 raft_consensus.cc:348] T 83e28eea13a14c64b9d3a4ed5a6b7ed2 P 600d04a3960143f89a8e1bee5bcd65e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2972efc435eb470c88b7c8689c4b96e1" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 38593 } } peers { permanent_uuid: "524610df19f84cfd88b4b7d4c3aaacdc" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 35655 } } peers { permanent_uuid: "600d04a3960143f89a8e1bee5bcd65e7" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 45797 } }
I20260810 11:30:22.293579 10557 raft_consensus.cc:374] T 83e28eea13a14c64b9d3a4ed5a6b7ed2 P 600d04a3960143f89a8e1bee5bcd65e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:30:22.293715 10557 raft_consensus.cc:729] T 83e28eea13a14c64b9d3a4ed5a6b7ed2 P 600d04a3960143f89a8e1bee5bcd65e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 600d04a3960143f89a8e1bee5bcd65e7, State: Initialized, Role: FOLLOWER
I20260810 11:30:22.294447 10557 consensus_queue.cc:260] T 83e28eea13a14c64b9d3a4ed5a6b7ed2 P 600d04a3960143f89a8e1bee5bcd65e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2972efc435eb470c88b7c8689c4b96e1" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 38593 } } peers { permanent_uuid: "524610df19f84cfd88b4b7d4c3aaacdc" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 35655 } } peers { permanent_uuid: "600d04a3960143f89a8e1bee5bcd65e7" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 45797 } }
I20260810 11:30:22.295626 10392 heartbeater.cc:503] Master 127.8.18.254:40395 was elected leader, sending a full tablet report...
I20260810 11:30:22.296398 10557 ts_tablet_manager.cc:1428] T 83e28eea13a14c64b9d3a4ed5a6b7ed2 P 600d04a3960143f89a8e1bee5bcd65e7: Time spent starting tablet: real 0.012s user 0.013s sys 0.000s
I20260810 11:30:22.367652 10561 raft_consensus.cc:482] T 83e28eea13a14c64b9d3a4ed5a6b7ed2 P 524610df19f84cfd88b4b7d4c3aaacdc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 11:30:22.368053 10561 raft_consensus.cc:504] T 83e28eea13a14c64b9d3a4ed5a6b7ed2 P 524610df19f84cfd88b4b7d4c3aaacdc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2972efc435eb470c88b7c8689c4b96e1" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 38593 } } peers { permanent_uuid: "524610df19f84cfd88b4b7d4c3aaacdc" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 35655 } } peers { permanent_uuid: "600d04a3960143f89a8e1bee5bcd65e7" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 45797 } }
I20260810 11:30:22.369936 10561 leader_election.cc:288] T 83e28eea13a14c64b9d3a4ed5a6b7ed2 P 524610df19f84cfd88b4b7d4c3aaacdc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2972efc435eb470c88b7c8689c4b96e1 (127.8.18.195:38593), 600d04a3960143f89a8e1bee5bcd65e7 (127.8.18.194:45797)
W20260810 11:30:22.374729 10537 tablet.cc:2369] T 83e28eea13a14c64b9d3a4ed5a6b7ed2 P 2972efc435eb470c88b7c8689c4b96e1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 11:30:22.376796 10554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:22.370517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.193:60085 (local address 127.8.18.194:45797)
0810 11:30:22.370768 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0810 11:30:22.370774 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:30:22.371105 (+ 331us) server_negotiation.cc:407] Connection header received
0810 11:30:22.371351 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:30:22.371364 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:30:22.371426 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:30:22.371528 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:30:22.372169 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:22.372930 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:22.374808 (+ 1878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:22.375703 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:22.375966 (+ 263us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:22.376050 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0810 11:30:22.376353 (+ 303us) server_negotiation.cc:299] Negotiation successful
0810 11:30:22.376493 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260810 11:30:22.376837 10564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:22.370277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.194:45797 (local address 127.8.18.193:60085)
0810 11:30:22.370980 (+ 703us) negotiation.cc:107] Waiting for socket to connect
0810 11:30:22.370997 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 11:30:22.371186 (+ 189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:30:22.371559 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:30:22.371564 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:30:22.371603 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:30:22.372015 (+ 412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:30:22.372025 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:30:22.373070 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:30:22.373076 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:30:22.374623 (+ 1547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:30:22.374639 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:30:22.375993 (+ 1354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:30:22.376000 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:30:22.376176 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:22.376202 (+ 26us) client_negotiation.cc:769] Sending connection context
0810 11:30:22.376324 (+ 122us) client_negotiation.cc:240] Negotiation successful
0810 11:30:22.376473 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":592,"thread_start_us":358,"threads_started":1}
I20260810 11:30:22.378085 10565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:22.370277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.195:38593 (local address 127.8.18.193:44895)
0810 11:30:22.371715 (+ 1438us) negotiation.cc:107] Waiting for socket to connect
0810 11:30:22.371732 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 11:30:22.371927 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:30:22.372294 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:30:22.372299 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:30:22.372326 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:30:22.372669 (+ 343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:30:22.372679 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:30:22.373862 (+ 1183us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:30:22.373872 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:30:22.375961 (+ 2089us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:30:22.375979 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:30:22.377394 (+ 1415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:30:22.377401 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:30:22.377602 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:22.377629 (+ 27us) client_negotiation.cc:769] Sending connection context
0810 11:30:22.377711 (+ 82us) client_negotiation.cc:240] Negotiation successful
0810 11:30:22.377811 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1277,"mutex_wait_us":608,"thread_start_us":435,"threads_started":1}
I20260810 11:30:22.378252 10550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:22.370632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.193:44895 (local address 127.8.18.195:38593)
0810 11:30:22.370865 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0810 11:30:22.370871 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:30:22.371853 (+ 982us) server_negotiation.cc:407] Connection header received
0810 11:30:22.372088 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:30:22.372105 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:30:22.372169 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:30:22.372278 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:30:22.372838 (+ 560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:22.373705 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:22.376155 (+ 2450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:22.377174 (+ 1019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:22.377357 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:22.377480 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0810 11:30:22.377776 (+ 296us) server_negotiation.cc:299] Negotiation successful
0810 11:30:22.377936 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260810 11:30:22.378095 10346 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "83e28eea13a14c64b9d3a4ed5a6b7ed2" candidate_uuid: "524610df19f84cfd88b4b7d4c3aaacdc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "600d04a3960143f89a8e1bee5bcd65e7" is_pre_election: true
I20260810 11:30:22.378799 10346 raft_consensus.cc:2454] T 83e28eea13a14c64b9d3a4ed5a6b7ed2 P 600d04a3960143f89a8e1bee5bcd65e7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 524610df19f84cfd88b4b7d4c3aaacdc in term 0.
I20260810 11:30:22.379288 10484 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "83e28eea13a14c64b9d3a4ed5a6b7ed2" candidate_uuid: "524610df19f84cfd88b4b7d4c3aaacdc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2972efc435eb470c88b7c8689c4b96e1" is_pre_election: true
I20260810 11:30:22.379693 10130 leader_election.cc:302] T 83e28eea13a14c64b9d3a4ed5a6b7ed2 P 524610df19f84cfd88b4b7d4c3aaacdc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 524610df19f84cfd88b4b7d4c3aaacdc, 600d04a3960143f89a8e1bee5bcd65e7; no voters:
I20260810 11:30:22.379870 10484 raft_consensus.cc:2454] T 83e28eea13a14c64b9d3a4ed5a6b7ed2 P 2972efc435eb470c88b7c8689c4b96e1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 524610df19f84cfd88b4b7d4c3aaacdc in term 0.
I20260810 11:30:22.380206 10561 raft_consensus.cc:2781] T 83e28eea13a14c64b9d3a4ed5a6b7ed2 P 524610df19f84cfd88b4b7d4c3aaacdc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 11:30:22.380375 10561 raft_consensus.cc:482] T 83e28eea13a14c64b9d3a4ed5a6b7ed2 P 524610df19f84cfd88b4b7d4c3aaacdc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 11:30:22.380493 10561 raft_consensus.cc:3037] T 83e28eea13a14c64b9d3a4ed5a6b7ed2 P 524610df19f84cfd88b4b7d4c3aaacdc [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:30:22.382961 10561 raft_consensus.cc:504] T 83e28eea13a14c64b9d3a4ed5a6b7ed2 P 524610df19f84cfd88b4b7d4c3aaacdc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2972efc435eb470c88b7c8689c4b96e1" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 38593 } } peers { permanent_uuid: "524610df19f84cfd88b4b7d4c3aaacdc" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 35655 } } peers { permanent_uuid: "600d04a3960143f89a8e1bee5bcd65e7" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 45797 } }
I20260810 11:30:22.383711 10561 leader_election.cc:288] T 83e28eea13a14c64b9d3a4ed5a6b7ed2 P 524610df19f84cfd88b4b7d4c3aaacdc [CANDIDATE]: Term 1 election: Requested vote from peers 2972efc435eb470c88b7c8689c4b96e1 (127.8.18.195:38593), 600d04a3960143f89a8e1bee5bcd65e7 (127.8.18.194:45797)
I20260810 11:30:22.384399 10346 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "83e28eea13a14c64b9d3a4ed5a6b7ed2" candidate_uuid: "524610df19f84cfd88b4b7d4c3aaacdc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "600d04a3960143f89a8e1bee5bcd65e7"
I20260810 11:30:22.384433 10484 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "83e28eea13a14c64b9d3a4ed5a6b7ed2" candidate_uuid: "524610df19f84cfd88b4b7d4c3aaacdc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2972efc435eb470c88b7c8689c4b96e1"
I20260810 11:30:22.384677 10484 raft_consensus.cc:3037] T 83e28eea13a14c64b9d3a4ed5a6b7ed2 P 2972efc435eb470c88b7c8689c4b96e1 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:30:22.384671 10346 raft_consensus.cc:3037] T 83e28eea13a14c64b9d3a4ed5a6b7ed2 P 600d04a3960143f89a8e1bee5bcd65e7 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:30:22.387461 10484 raft_consensus.cc:2454] T 83e28eea13a14c64b9d3a4ed5a6b7ed2 P 2972efc435eb470c88b7c8689c4b96e1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 524610df19f84cfd88b4b7d4c3aaacdc in term 1.
I20260810 11:30:22.387555 10346 raft_consensus.cc:2454] T 83e28eea13a14c64b9d3a4ed5a6b7ed2 P 600d04a3960143f89a8e1bee5bcd65e7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 524610df19f84cfd88b4b7d4c3aaacdc in term 1.
I20260810 11:30:22.388134 10131 leader_election.cc:302] T 83e28eea13a14c64b9d3a4ed5a6b7ed2 P 524610df19f84cfd88b4b7d4c3aaacdc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2972efc435eb470c88b7c8689c4b96e1, 524610df19f84cfd88b4b7d4c3aaacdc; no voters:
I20260810 11:30:22.388509 10561 raft_consensus.cc:2781] T 83e28eea13a14c64b9d3a4ed5a6b7ed2 P 524610df19f84cfd88b4b7d4c3aaacdc [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:30:22.388983 10561 raft_consensus.cc:686] T 83e28eea13a14c64b9d3a4ed5a6b7ed2 P 524610df19f84cfd88b4b7d4c3aaacdc [term 1 LEADER]: Becoming Leader. State: Replica: 524610df19f84cfd88b4b7d4c3aaacdc, State: Running, Role: LEADER
I20260810 11:30:22.389637 10561 consensus_queue.cc:237] T 83e28eea13a14c64b9d3a4ed5a6b7ed2 P 524610df19f84cfd88b4b7d4c3aaacdc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2972efc435eb470c88b7c8689c4b96e1" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 38593 } } peers { permanent_uuid: "524610df19f84cfd88b4b7d4c3aaacdc" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 35655 } } peers { permanent_uuid: "600d04a3960143f89a8e1bee5bcd65e7" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 45797 } }
I20260810 11:30:22.394572 10038 catalog_manager.cc:5495] T 83e28eea13a14c64b9d3a4ed5a6b7ed2 P 524610df19f84cfd88b4b7d4c3aaacdc reported cstate change: term changed from 0 to 1, leader changed from <none> to 524610df19f84cfd88b4b7d4c3aaacdc (127.8.18.193). New cstate: current_term: 1 leader_uuid: "524610df19f84cfd88b4b7d4c3aaacdc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2972efc435eb470c88b7c8689c4b96e1" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 38593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "524610df19f84cfd88b4b7d4c3aaacdc" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 35655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "600d04a3960143f89a8e1bee5bcd65e7" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 45797 } health_report { overall_health: UNKNOWN } } }
Aug 10 11:30:22 dist-test-slave-srgv krb5kdc[9992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361419, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.254@KRBTEST.COM for kudu/127.8.18.254@KRBTEST.COM
I20260810 11:30:22.459924 10573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:22.444255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:40395 (local address 127.0.0.1:41822)
0810 11:30:22.444931 (+ 676us) negotiation.cc:107] Waiting for socket to connect
0810 11:30:22.444949 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 11:30:22.445093 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:30:22.445504 (+ 411us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:30:22.445508 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:30:22.446236 (+ 728us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:30:22.446535 (+ 299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:30:22.446546 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:30:22.447777 (+ 1231us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:30:22.447782 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:30:22.448943 (+ 1161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:30:22.448957 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:30:22.449809 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:30:22.449815 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:30:22.449948 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:22.450881 (+ 933us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:30:22.450901 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:30:22.453494 (+ 2593us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:30:22.457142 (+ 3648us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:30:22.457158 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:30:22.457164 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:30:22.457873 (+ 709us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:30:22.458266 (+ 393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:30:22.458270 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:30:22.458272 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:30:22.458449 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:30:22.458975 (+ 526us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:30:22.458981 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:30:22.459181 (+ 200us) client_negotiation.cc:769] Sending connection context
0810 11:30:22.459329 (+ 148us) client_negotiation.cc:240] Negotiation successful
0810 11:30:22.459565 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":551,"thread_start_us":385,"threads_started":1}
I20260810 11:30:22.459933 10422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:22.444212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41822 (local address 127.8.18.254:40395)
0810 11:30:22.444423 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0810 11:30:22.444428 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:30:22.445055 (+ 627us) server_negotiation.cc:407] Connection header received
0810 11:30:22.445203 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:30:22.445207 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:30:22.445279 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:30:22.445367 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:30:22.446689 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:22.447580 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:22.449086 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:22.449613 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:22.449713 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:22.453740 (+ 4027us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:30:22.453746 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:30:22.453750 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:30:22.453787 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:30:22.456934 (+ 3147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:22.458005 (+ 1071us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:22.458009 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:22.458012 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:22.458143 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:22.458556 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:22.458559 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:22.458561 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:22.458847 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:30:22.458962 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0810 11:30:22.459501 (+ 539us) server_negotiation.cc:299] Negotiation successful
0810 11:30:22.459693 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260810 11:30:22.466109 10038 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.8.18.254@KRBTEST.COM'} at 127.0.0.1:41822:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260810 11:30:22.467160 10038 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260810 11:30:22.468827 10253 tablet.cc:2369] T 83e28eea13a14c64b9d3a4ed5a6b7ed2 P 524610df19f84cfd88b4b7d4c3aaacdc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 11:30:22.475404 10179 tablet_service.cc:1467] Processing CreateTablet for tablet f05779abb5fd4d92b007f9fe40efc1e7 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=bb02b6366ce34e1488bf52b16eb85565]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 11:30:22.475536 10321 tablet_service.cc:1467] Processing CreateTablet for tablet f05779abb5fd4d92b007f9fe40efc1e7 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=bb02b6366ce34e1488bf52b16eb85565]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 11:30:22.476526 10321 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f05779abb5fd4d92b007f9fe40efc1e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:30:22.476562 10179 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f05779abb5fd4d92b007f9fe40efc1e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:30:22.481819 10464 tablet_service.cc:1467] Processing CreateTablet for tablet f05779abb5fd4d92b007f9fe40efc1e7 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=bb02b6366ce34e1488bf52b16eb85565]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 11:30:22.482657 10557 tablet_bootstrap.cc:492] T f05779abb5fd4d92b007f9fe40efc1e7 P 600d04a3960143f89a8e1bee5bcd65e7: Bootstrap starting.
I20260810 11:30:22.482654 10556 tablet_bootstrap.cc:492] T f05779abb5fd4d92b007f9fe40efc1e7 P 524610df19f84cfd88b4b7d4c3aaacdc: Bootstrap starting.
I20260810 11:30:22.482908 10464 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f05779abb5fd4d92b007f9fe40efc1e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:30:22.484045 10556 tablet_bootstrap.cc:654] T f05779abb5fd4d92b007f9fe40efc1e7 P 524610df19f84cfd88b4b7d4c3aaacdc: Neither blocks nor log segments found. Creating new log.
I20260810 11:30:22.484418 10557 tablet_bootstrap.cc:654] T f05779abb5fd4d92b007f9fe40efc1e7 P 600d04a3960143f89a8e1bee5bcd65e7: Neither blocks nor log segments found. Creating new log.
I20260810 11:30:22.492676 10555 tablet_bootstrap.cc:492] T f05779abb5fd4d92b007f9fe40efc1e7 P 2972efc435eb470c88b7c8689c4b96e1: Bootstrap starting.
I20260810 11:30:22.492669 10557 tablet_bootstrap.cc:492] T f05779abb5fd4d92b007f9fe40efc1e7 P 600d04a3960143f89a8e1bee5bcd65e7: No bootstrap required, opened a new log
I20260810 11:30:22.492683 10556 tablet_bootstrap.cc:492] T f05779abb5fd4d92b007f9fe40efc1e7 P 524610df19f84cfd88b4b7d4c3aaacdc: No bootstrap required, opened a new log
I20260810 11:30:22.492861 10557 ts_tablet_manager.cc:1397] T f05779abb5fd4d92b007f9fe40efc1e7 P 600d04a3960143f89a8e1bee5bcd65e7: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.004s
I20260810 11:30:22.492861 10556 ts_tablet_manager.cc:1397] T f05779abb5fd4d92b007f9fe40efc1e7 P 524610df19f84cfd88b4b7d4c3aaacdc: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.004s
I20260810 11:30:22.493738 10557 raft_consensus.cc:348] T f05779abb5fd4d92b007f9fe40efc1e7 P 600d04a3960143f89a8e1bee5bcd65e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "524610df19f84cfd88b4b7d4c3aaacdc" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 35655 } } peers { permanent_uuid: "600d04a3960143f89a8e1bee5bcd65e7" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 45797 } } peers { permanent_uuid: "2972efc435eb470c88b7c8689c4b96e1" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 38593 } }
I20260810 11:30:22.493744 10556 raft_consensus.cc:348] T f05779abb5fd4d92b007f9fe40efc1e7 P 524610df19f84cfd88b4b7d4c3aaacdc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "524610df19f84cfd88b4b7d4c3aaacdc" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 35655 } } peers { permanent_uuid: "600d04a3960143f89a8e1bee5bcd65e7" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 45797 } } peers { permanent_uuid: "2972efc435eb470c88b7c8689c4b96e1" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 38593 } }
I20260810 11:30:22.494012 10557 raft_consensus.cc:374] T f05779abb5fd4d92b007f9fe40efc1e7 P 600d04a3960143f89a8e1bee5bcd65e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:30:22.494045 10556 raft_consensus.cc:374] T f05779abb5fd4d92b007f9fe40efc1e7 P 524610df19f84cfd88b4b7d4c3aaacdc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:30:22.494127 10557 raft_consensus.cc:729] T f05779abb5fd4d92b007f9fe40efc1e7 P 600d04a3960143f89a8e1bee5bcd65e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 600d04a3960143f89a8e1bee5bcd65e7, State: Initialized, Role: FOLLOWER
I20260810 11:30:22.494136 10556 raft_consensus.cc:729] T f05779abb5fd4d92b007f9fe40efc1e7 P 524610df19f84cfd88b4b7d4c3aaacdc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 524610df19f84cfd88b4b7d4c3aaacdc, State: Initialized, Role: FOLLOWER
I20260810 11:30:22.494484 10555 tablet_bootstrap.cc:654] T f05779abb5fd4d92b007f9fe40efc1e7 P 2972efc435eb470c88b7c8689c4b96e1: Neither blocks nor log segments found. Creating new log.
I20260810 11:30:22.494382 10556 consensus_queue.cc:260] T f05779abb5fd4d92b007f9fe40efc1e7 P 524610df19f84cfd88b4b7d4c3aaacdc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "524610df19f84cfd88b4b7d4c3aaacdc" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 35655 } } peers { permanent_uuid: "600d04a3960143f89a8e1bee5bcd65e7" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 45797 } } peers { permanent_uuid: "2972efc435eb470c88b7c8689c4b96e1" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 38593 } }
I20260810 11:30:22.494382 10557 consensus_queue.cc:260] T f05779abb5fd4d92b007f9fe40efc1e7 P 600d04a3960143f89a8e1bee5bcd65e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "524610df19f84cfd88b4b7d4c3aaacdc" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 35655 } } peers { permanent_uuid: "600d04a3960143f89a8e1bee5bcd65e7" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 45797 } } peers { permanent_uuid: "2972efc435eb470c88b7c8689c4b96e1" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 38593 } }
I20260810 11:30:22.494979 10557 ts_tablet_manager.cc:1428] T f05779abb5fd4d92b007f9fe40efc1e7 P 600d04a3960143f89a8e1bee5bcd65e7: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260810 11:30:22.495004 10556 ts_tablet_manager.cc:1428] T f05779abb5fd4d92b007f9fe40efc1e7 P 524610df19f84cfd88b4b7d4c3aaacdc: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260810 11:30:22.495393 10561 tablet_replica.cc:439] T f05779abb5fd4d92b007f9fe40efc1e7 P 524610df19f84cfd88b4b7d4c3aaacdc: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "524610df19f84cfd88b4b7d4c3aaacdc" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 35655 } } peers { permanent_uuid: "600d04a3960143f89a8e1bee5bcd65e7" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 45797 } } peers { permanent_uuid: "2972efc435eb470c88b7c8689c4b96e1" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 38593 } } }
I20260810 11:30:22.495688 10561 tablet_replica.cc:442] T f05779abb5fd4d92b007f9fe40efc1e7 P 524610df19f84cfd88b4b7d4c3aaacdc: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 11:30:22.496686 10563 tablet_replica.cc:439] T f05779abb5fd4d92b007f9fe40efc1e7 P 600d04a3960143f89a8e1bee5bcd65e7: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "524610df19f84cfd88b4b7d4c3aaacdc" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 35655 } } peers { permanent_uuid: "600d04a3960143f89a8e1bee5bcd65e7" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 45797 } } peers { permanent_uuid: "2972efc435eb470c88b7c8689c4b96e1" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 38593 } } }
I20260810 11:30:22.497097 10563 tablet_replica.cc:442] T f05779abb5fd4d92b007f9fe40efc1e7 P 600d04a3960143f89a8e1bee5bcd65e7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 11:30:22.503003 10555 tablet_bootstrap.cc:492] T f05779abb5fd4d92b007f9fe40efc1e7 P 2972efc435eb470c88b7c8689c4b96e1: No bootstrap required, opened a new log
I20260810 11:30:22.503228 10555 ts_tablet_manager.cc:1397] T f05779abb5fd4d92b007f9fe40efc1e7 P 2972efc435eb470c88b7c8689c4b96e1: Time spent bootstrapping tablet: real 0.011s user 0.006s sys 0.003s
I20260810 11:30:22.504022 10555 raft_consensus.cc:348] T f05779abb5fd4d92b007f9fe40efc1e7 P 2972efc435eb470c88b7c8689c4b96e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "524610df19f84cfd88b4b7d4c3aaacdc" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 35655 } } peers { permanent_uuid: "600d04a3960143f89a8e1bee5bcd65e7" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 45797 } } peers { permanent_uuid: "2972efc435eb470c88b7c8689c4b96e1" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 38593 } }
I20260810 11:30:22.504226 10555 raft_consensus.cc:374] T f05779abb5fd4d92b007f9fe40efc1e7 P 2972efc435eb470c88b7c8689c4b96e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:30:22.504289 10555 raft_consensus.cc:729] T f05779abb5fd4d92b007f9fe40efc1e7 P 2972efc435eb470c88b7c8689c4b96e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2972efc435eb470c88b7c8689c4b96e1, State: Initialized, Role: FOLLOWER
I20260810 11:30:22.504525 10555 consensus_queue.cc:260] T f05779abb5fd4d92b007f9fe40efc1e7 P 2972efc435eb470c88b7c8689c4b96e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "524610df19f84cfd88b4b7d4c3aaacdc" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 35655 } } peers { permanent_uuid: "600d04a3960143f89a8e1bee5bcd65e7" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 45797 } } peers { permanent_uuid: "2972efc435eb470c88b7c8689c4b96e1" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 38593 } }
I20260810 11:30:22.505232 10555 ts_tablet_manager.cc:1428] T f05779abb5fd4d92b007f9fe40efc1e7 P 2972efc435eb470c88b7c8689c4b96e1: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260810 11:30:22.505070 10562 tablet_replica.cc:439] T f05779abb5fd4d92b007f9fe40efc1e7 P 2972efc435eb470c88b7c8689c4b96e1: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "524610df19f84cfd88b4b7d4c3aaacdc" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 35655 } } peers { permanent_uuid: "600d04a3960143f89a8e1bee5bcd65e7" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 45797 } } peers { permanent_uuid: "2972efc435eb470c88b7c8689c4b96e1" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 38593 } } }
I20260810 11:30:22.505553 10562 tablet_replica.cc:442] T f05779abb5fd4d92b007f9fe40efc1e7 P 2972efc435eb470c88b7c8689c4b96e1: This TxnStatusTablet replica's current role is: FOLLOWER
W20260810 11:30:22.540307 10393 tablet.cc:2369] T 83e28eea13a14c64b9d3a4ed5a6b7ed2 P 600d04a3960143f89a8e1bee5bcd65e7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 11:30:22.546979 10563 raft_consensus.cc:482] T f05779abb5fd4d92b007f9fe40efc1e7 P 600d04a3960143f89a8e1bee5bcd65e7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 11:30:22.547273 10563 raft_consensus.cc:504] T f05779abb5fd4d92b007f9fe40efc1e7 P 600d04a3960143f89a8e1bee5bcd65e7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "524610df19f84cfd88b4b7d4c3aaacdc" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 35655 } } peers { permanent_uuid: "600d04a3960143f89a8e1bee5bcd65e7" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 45797 } } peers { permanent_uuid: "2972efc435eb470c88b7c8689c4b96e1" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 38593 } }
I20260810 11:30:22.549305 10563 leader_election.cc:288] T f05779abb5fd4d92b007f9fe40efc1e7 P 600d04a3960143f89a8e1bee5bcd65e7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 524610df19f84cfd88b4b7d4c3aaacdc (127.8.18.193:35655), 2972efc435eb470c88b7c8689c4b96e1 (127.8.18.195:38593)
I20260810 11:30:22.557925 10552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:22.550360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.194:34891 (local address 127.8.18.193:35655)
0810 11:30:22.550593 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0810 11:30:22.550603 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 11:30:22.551064 (+ 461us) server_negotiation.cc:407] Connection header received
0810 11:30:22.551384 (+ 320us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:30:22.551394 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:30:22.551504 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:30:22.551625 (+ 121us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:30:22.552506 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:22.553486 (+ 980us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:22.555828 (+ 2342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:22.556786 (+ 958us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:22.556932 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:22.556992 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0810 11:30:22.557471 (+ 479us) server_negotiation.cc:299] Negotiation successful
0810 11:30:22.557649 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260810 11:30:22.557997 10577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:22.549796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.193:35655 (local address 127.8.18.194:34891)
0810 11:30:22.550914 (+ 1118us) negotiation.cc:107] Waiting for socket to connect
0810 11:30:22.550937 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0810 11:30:22.551200 (+ 263us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:30:22.551698 (+ 498us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:30:22.551702 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:30:22.551733 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:30:22.552312 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:30:22.552326 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:30:22.553638 (+ 1312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:30:22.553645 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:30:22.555638 (+ 1993us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:30:22.555655 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:30:22.557037 (+ 1382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:30:22.557044 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:30:22.557231 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:22.557280 (+ 49us) client_negotiation.cc:769] Sending connection context
0810 11:30:22.557388 (+ 108us) client_negotiation.cc:240] Negotiation successful
0810 11:30:22.557577 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":967,"mutex_wait_us":20,"thread_start_us":1211,"threads_started":1}
I20260810 11:30:22.558497 10578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:22.549790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.195:38593 (local address 127.8.18.194:57623)
0810 11:30:22.551411 (+ 1621us) negotiation.cc:107] Waiting for socket to connect
0810 11:30:22.551430 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 11:30:22.551573 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:30:22.551994 (+ 421us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:30:22.551997 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:30:22.552022 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:30:22.552531 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:30:22.552542 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:30:22.554377 (+ 1835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:30:22.554382 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:30:22.556005 (+ 1623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:30:22.556021 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:30:22.557451 (+ 1430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:30:22.557458 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:30:22.557645 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:22.557678 (+ 33us) client_negotiation.cc:769] Sending connection context
0810 11:30:22.557786 (+ 108us) client_negotiation.cc:240] Negotiation successful
0810 11:30:22.557930 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1448,"mutex_wait_us":90,"thread_start_us":773,"threads_started":1}
I20260810 11:30:22.559067 10199 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f05779abb5fd4d92b007f9fe40efc1e7" candidate_uuid: "600d04a3960143f89a8e1bee5bcd65e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "524610df19f84cfd88b4b7d4c3aaacdc" is_pre_election: true
I20260810 11:30:22.559563 10550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:22.549824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.194:57623 (local address 127.8.18.195:38593)
0810 11:30:22.550015 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0810 11:30:22.550022 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:30:22.551663 (+ 1641us) server_negotiation.cc:407] Connection header received
0810 11:30:22.551753 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:30:22.551760 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:30:22.551851 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:30:22.551971 (+ 120us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:30:22.552739 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:22.554170 (+ 1431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:22.556228 (+ 2058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:22.557150 (+ 922us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:22.557331 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:22.557444 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0810 11:30:22.557843 (+ 399us) server_negotiation.cc:299] Negotiation successful
0810 11:30:22.558379 (+ 536us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260810 11:30:22.559593 10199 raft_consensus.cc:2454] T f05779abb5fd4d92b007f9fe40efc1e7 P 524610df19f84cfd88b4b7d4c3aaacdc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 600d04a3960143f89a8e1bee5bcd65e7 in term 0.
I20260810 11:30:22.560359 10273 leader_election.cc:302] T f05779abb5fd4d92b007f9fe40efc1e7 P 600d04a3960143f89a8e1bee5bcd65e7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 524610df19f84cfd88b4b7d4c3aaacdc, 600d04a3960143f89a8e1bee5bcd65e7; no voters:
I20260810 11:30:22.560423 10484 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f05779abb5fd4d92b007f9fe40efc1e7" candidate_uuid: "600d04a3960143f89a8e1bee5bcd65e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2972efc435eb470c88b7c8689c4b96e1" is_pre_election: true
I20260810 11:30:22.560736 10484 raft_consensus.cc:2454] T f05779abb5fd4d92b007f9fe40efc1e7 P 2972efc435eb470c88b7c8689c4b96e1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 600d04a3960143f89a8e1bee5bcd65e7 in term 0.
I20260810 11:30:22.560885 10563 raft_consensus.cc:2781] T f05779abb5fd4d92b007f9fe40efc1e7 P 600d04a3960143f89a8e1bee5bcd65e7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 11:30:22.561028 10563 raft_consensus.cc:482] T f05779abb5fd4d92b007f9fe40efc1e7 P 600d04a3960143f89a8e1bee5bcd65e7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 11:30:22.561132 10563 raft_consensus.cc:3037] T f05779abb5fd4d92b007f9fe40efc1e7 P 600d04a3960143f89a8e1bee5bcd65e7 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:30:22.563577 10563 raft_consensus.cc:504] T f05779abb5fd4d92b007f9fe40efc1e7 P 600d04a3960143f89a8e1bee5bcd65e7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "524610df19f84cfd88b4b7d4c3aaacdc" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 35655 } } peers { permanent_uuid: "600d04a3960143f89a8e1bee5bcd65e7" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 45797 } } peers { permanent_uuid: "2972efc435eb470c88b7c8689c4b96e1" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 38593 } }
I20260810 11:30:22.564314 10563 leader_election.cc:288] T f05779abb5fd4d92b007f9fe40efc1e7 P 600d04a3960143f89a8e1bee5bcd65e7 [CANDIDATE]: Term 1 election: Requested vote from peers 524610df19f84cfd88b4b7d4c3aaacdc (127.8.18.193:35655), 2972efc435eb470c88b7c8689c4b96e1 (127.8.18.195:38593)
I20260810 11:30:22.564903 10484 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f05779abb5fd4d92b007f9fe40efc1e7" candidate_uuid: "600d04a3960143f89a8e1bee5bcd65e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2972efc435eb470c88b7c8689c4b96e1"
I20260810 11:30:22.564917 10199 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f05779abb5fd4d92b007f9fe40efc1e7" candidate_uuid: "600d04a3960143f89a8e1bee5bcd65e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "524610df19f84cfd88b4b7d4c3aaacdc"
I20260810 11:30:22.565157 10199 raft_consensus.cc:3037] T f05779abb5fd4d92b007f9fe40efc1e7 P 524610df19f84cfd88b4b7d4c3aaacdc [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:30:22.565162 10484 raft_consensus.cc:3037] T f05779abb5fd4d92b007f9fe40efc1e7 P 2972efc435eb470c88b7c8689c4b96e1 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:30:22.567574 10484 raft_consensus.cc:2454] T f05779abb5fd4d92b007f9fe40efc1e7 P 2972efc435eb470c88b7c8689c4b96e1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 600d04a3960143f89a8e1bee5bcd65e7 in term 1.
I20260810 11:30:22.567775 10199 raft_consensus.cc:2454] T f05779abb5fd4d92b007f9fe40efc1e7 P 524610df19f84cfd88b4b7d4c3aaacdc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 600d04a3960143f89a8e1bee5bcd65e7 in term 1.
I20260810 11:30:22.568296 10273 leader_election.cc:302] T f05779abb5fd4d92b007f9fe40efc1e7 P 600d04a3960143f89a8e1bee5bcd65e7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 524610df19f84cfd88b4b7d4c3aaacdc, 600d04a3960143f89a8e1bee5bcd65e7; no voters:
I20260810 11:30:22.568689 10563 raft_consensus.cc:2781] T f05779abb5fd4d92b007f9fe40efc1e7 P 600d04a3960143f89a8e1bee5bcd65e7 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:30:22.569238 10563 raft_consensus.cc:686] T f05779abb5fd4d92b007f9fe40efc1e7 P 600d04a3960143f89a8e1bee5bcd65e7 [term 1 LEADER]: Becoming Leader. State: Replica: 600d04a3960143f89a8e1bee5bcd65e7, State: Running, Role: LEADER
I20260810 11:30:22.569799 10563 consensus_queue.cc:237] T f05779abb5fd4d92b007f9fe40efc1e7 P 600d04a3960143f89a8e1bee5bcd65e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "524610df19f84cfd88b4b7d4c3aaacdc" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 35655 } } peers { permanent_uuid: "600d04a3960143f89a8e1bee5bcd65e7" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 45797 } } peers { permanent_uuid: "2972efc435eb470c88b7c8689c4b96e1" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 38593 } }
I20260810 11:30:22.573369 10579 tablet_replica.cc:439] T f05779abb5fd4d92b007f9fe40efc1e7 P 600d04a3960143f89a8e1bee5bcd65e7: TxnStatusTablet state changed. Reason: New leader 600d04a3960143f89a8e1bee5bcd65e7. Latest consensus state: current_term: 1 leader_uuid: "600d04a3960143f89a8e1bee5bcd65e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "524610df19f84cfd88b4b7d4c3aaacdc" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 35655 } } peers { permanent_uuid: "600d04a3960143f89a8e1bee5bcd65e7" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 45797 } } peers { permanent_uuid: "2972efc435eb470c88b7c8689c4b96e1" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 38593 } } }
I20260810 11:30:22.573884 10579 tablet_replica.cc:442] T f05779abb5fd4d92b007f9fe40efc1e7 P 600d04a3960143f89a8e1bee5bcd65e7: This TxnStatusTablet replica's current role is: LEADER
I20260810 11:30:22.574717 10581 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260810 11:30:22.575035 10581 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260810 11:30:22.575039 10037 catalog_manager.cc:5495] T f05779abb5fd4d92b007f9fe40efc1e7 P 600d04a3960143f89a8e1bee5bcd65e7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 600d04a3960143f89a8e1bee5bcd65e7 (127.8.18.194). New cstate: current_term: 1 leader_uuid: "600d04a3960143f89a8e1bee5bcd65e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "524610df19f84cfd88b4b7d4c3aaacdc" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 35655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "600d04a3960143f89a8e1bee5bcd65e7" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 45797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2972efc435eb470c88b7c8689c4b96e1" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 38593 } health_report { overall_health: UNKNOWN } } }
I20260810 11:30:22.594871 10573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:22.588703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.194:45797 (local address 127.0.0.1:47420)
0810 11:30:22.588966 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0810 11:30:22.588990 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0810 11:30:22.589204 (+ 214us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:30:22.589777 (+ 573us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:30:22.589781 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:30:22.589794 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 11:30:22.589996 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:30:22.590005 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:30:22.591113 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:30:22.591119 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:30:22.592120 (+ 1001us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:30:22.592133 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:30:22.592939 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:30:22.592946 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:30:22.593151 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:22.593218 (+ 67us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 11:30:22.594254 (+ 1036us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 11:30:22.594281 (+ 27us) client_negotiation.cc:769] Sending connection context
0810 11:30:22.594371 (+ 90us) client_negotiation.cc:240] Negotiation successful
0810 11:30:22.594543 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260810 11:30:22.594847 10554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:22.588815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47420 (local address 127.8.18.194:45797)
0810 11:30:22.588993 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0810 11:30:22.588998 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:30:22.589090 (+ 92us) server_negotiation.cc:407] Connection header received
0810 11:30:22.589459 (+ 369us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:30:22.589465 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:30:22.589589 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:30:22.589756 (+ 167us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 11:30:22.590135 (+ 379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:22.590965 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:22.592270 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:22.592715 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:22.592912 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:22.593501 (+ 589us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 11:30:22.594132 (+ 631us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 11:30:22.594305 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0810 11:30:22.594396 (+ 91us) server_negotiation.cc:299] Negotiation successful
0810 11:30:22.594489 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260810 11:30:22.604049 10199 raft_consensus.cc:1261] T f05779abb5fd4d92b007f9fe40efc1e7 P 524610df19f84cfd88b4b7d4c3aaacdc [term 1 FOLLOWER]: Refusing update from remote peer 600d04a3960143f89a8e1bee5bcd65e7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 11:30:22.604137 10484 raft_consensus.cc:1261] T f05779abb5fd4d92b007f9fe40efc1e7 P 2972efc435eb470c88b7c8689c4b96e1 [term 1 FOLLOWER]: Refusing update from remote peer 600d04a3960143f89a8e1bee5bcd65e7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 11:30:22.605181 10563 consensus_queue.cc:1035] T f05779abb5fd4d92b007f9fe40efc1e7 P 600d04a3960143f89a8e1bee5bcd65e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "524610df19f84cfd88b4b7d4c3aaacdc" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 35655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 11:30:22.605680 10579 consensus_queue.cc:1035] T f05779abb5fd4d92b007f9fe40efc1e7 P 600d04a3960143f89a8e1bee5bcd65e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2972efc435eb470c88b7c8689c4b96e1" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 38593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 11:30:22.614449 10561 tablet_replica.cc:439] T f05779abb5fd4d92b007f9fe40efc1e7 P 524610df19f84cfd88b4b7d4c3aaacdc: TxnStatusTablet state changed. Reason: New leader 600d04a3960143f89a8e1bee5bcd65e7. Latest consensus state: current_term: 1 leader_uuid: "600d04a3960143f89a8e1bee5bcd65e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "524610df19f84cfd88b4b7d4c3aaacdc" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 35655 } } peers { permanent_uuid: "600d04a3960143f89a8e1bee5bcd65e7" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 45797 } } peers { permanent_uuid: "2972efc435eb470c88b7c8689c4b96e1" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 38593 } } }
I20260810 11:30:22.614878 10561 tablet_replica.cc:442] T f05779abb5fd4d92b007f9fe40efc1e7 P 524610df19f84cfd88b4b7d4c3aaacdc: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 11:30:22.617159 10584 mvcc.cc:205] Tried to move back new op lower bound from 7316936386975399936 to 7316936386849878016. Current Snapshot: MvccSnapshot[applied={T|T < 7316936386975399936}]
I20260810 11:30:22.619273 10585 mvcc.cc:205] Tried to move back new op lower bound from 7316936386975399936 to 7316936386849878016. Current Snapshot: MvccSnapshot[applied={T|T < 7316936386975399936}]
I20260810 11:30:22.620193 10563 tablet_replica.cc:439] T f05779abb5fd4d92b007f9fe40efc1e7 P 600d04a3960143f89a8e1bee5bcd65e7: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "600d04a3960143f89a8e1bee5bcd65e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "524610df19f84cfd88b4b7d4c3aaacdc" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 35655 } } peers { permanent_uuid: "600d04a3960143f89a8e1bee5bcd65e7" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 45797 } } peers { permanent_uuid: "2972efc435eb470c88b7c8689c4b96e1" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 38593 } } }
I20260810 11:30:22.620661 10563 tablet_replica.cc:442] T f05779abb5fd4d92b007f9fe40efc1e7 P 600d04a3960143f89a8e1bee5bcd65e7: This TxnStatusTablet replica's current role is: LEADER
I20260810 11:30:22.621140 10561 tablet_replica.cc:439] T f05779abb5fd4d92b007f9fe40efc1e7 P 524610df19f84cfd88b4b7d4c3aaacdc: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "600d04a3960143f89a8e1bee5bcd65e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "524610df19f84cfd88b4b7d4c3aaacdc" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 35655 } } peers { permanent_uuid: "600d04a3960143f89a8e1bee5bcd65e7" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 45797 } } peers { permanent_uuid: "2972efc435eb470c88b7c8689c4b96e1" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 38593 } } }
I20260810 11:30:22.621490 10561 tablet_replica.cc:442] T f05779abb5fd4d92b007f9fe40efc1e7 P 524610df19f84cfd88b4b7d4c3aaacdc: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 11:30:22.628295 10563 tablet_replica.cc:439] T f05779abb5fd4d92b007f9fe40efc1e7 P 600d04a3960143f89a8e1bee5bcd65e7: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "600d04a3960143f89a8e1bee5bcd65e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "524610df19f84cfd88b4b7d4c3aaacdc" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 35655 } } peers { permanent_uuid: "600d04a3960143f89a8e1bee5bcd65e7" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 45797 } } peers { permanent_uuid: "2972efc435eb470c88b7c8689c4b96e1" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 38593 } } }
I20260810 11:30:22.628623 10563 tablet_replica.cc:442] T f05779abb5fd4d92b007f9fe40efc1e7 P 600d04a3960143f89a8e1bee5bcd65e7: This TxnStatusTablet replica's current role is: LEADER
I20260810 11:30:22.628865 10562 tablet_replica.cc:439] T f05779abb5fd4d92b007f9fe40efc1e7 P 2972efc435eb470c88b7c8689c4b96e1: TxnStatusTablet state changed. Reason: New leader 600d04a3960143f89a8e1bee5bcd65e7. Latest consensus state: current_term: 1 leader_uuid: "600d04a3960143f89a8e1bee5bcd65e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "524610df19f84cfd88b4b7d4c3aaacdc" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 35655 } } peers { permanent_uuid: "600d04a3960143f89a8e1bee5bcd65e7" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 45797 } } peers { permanent_uuid: "2972efc435eb470c88b7c8689c4b96e1" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 38593 } } }
I20260810 11:30:22.629146 10562 tablet_replica.cc:442] T f05779abb5fd4d92b007f9fe40efc1e7 P 2972efc435eb470c88b7c8689c4b96e1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 11:30:22.632277 10562 tablet_replica.cc:439] T f05779abb5fd4d92b007f9fe40efc1e7 P 2972efc435eb470c88b7c8689c4b96e1: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "600d04a3960143f89a8e1bee5bcd65e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "524610df19f84cfd88b4b7d4c3aaacdc" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 35655 } } peers { permanent_uuid: "600d04a3960143f89a8e1bee5bcd65e7" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 45797 } } peers { permanent_uuid: "2972efc435eb470c88b7c8689c4b96e1" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 38593 } } }
I20260810 11:30:22.632543 10562 tablet_replica.cc:442] T f05779abb5fd4d92b007f9fe40efc1e7 P 2972efc435eb470c88b7c8689c4b96e1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 11:30:22.648797 10552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:22.641783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47366 (local address 127.8.18.193:35655)
0810 11:30:22.642012 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0810 11:30:22.642019 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:30:22.642196 (+ 177us) server_negotiation.cc:407] Connection header received
0810 11:30:22.642417 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:30:22.642424 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:30:22.642561 (+ 137us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:30:22.642755 (+ 194us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 11:30:22.643100 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:22.644299 (+ 1199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:22.645797 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:22.646452 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:22.646600 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:22.647126 (+ 526us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 11:30:22.647950 (+ 824us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 11:30:22.648222 (+ 272us) server_negotiation.cc:1035] Waiting for connection context
0810 11:30:22.648341 (+ 119us) server_negotiation.cc:299] Negotiation successful
0810 11:30:22.648449 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260810 11:30:22.667510 10554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:22.661231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.193:47081 (local address 127.8.18.194:45797)
0810 11:30:22.661498 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0810 11:30:22.661506 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:30:22.661907 (+ 401us) server_negotiation.cc:407] Connection header received
0810 11:30:22.662147 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:30:22.662159 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:30:22.662248 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:30:22.662351 (+ 103us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 11:30:22.663024 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:22.663837 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:22.665312 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:22.665973 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:22.666136 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:22.666751 (+ 615us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 11:30:22.666866 (+ 115us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 11:30:22.666952 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0810 11:30:22.667201 (+ 249us) server_negotiation.cc:299] Negotiation successful
0810 11:30:22.667278 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260810 11:30:22.667701 10595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:22.661096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.194:45797 (local address 127.8.18.193:47081)
0810 11:30:22.661793 (+ 697us) negotiation.cc:107] Waiting for socket to connect
0810 11:30:22.661810 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 11:30:22.661992 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:30:22.662372 (+ 380us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:30:22.662377 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:30:22.662390 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 11:30:22.662847 (+ 457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:30:22.662859 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:30:22.664001 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:30:22.664021 (+ 20us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:30:22.665154 (+ 1133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:30:22.665167 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:30:22.666211 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:30:22.666218 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:30:22.666410 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:22.666501 (+ 91us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 11:30:22.666987 (+ 486us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 11:30:22.666999 (+ 12us) client_negotiation.cc:769] Sending connection context
0810 11:30:22.667109 (+ 110us) client_negotiation.cc:240] Negotiation successful
0810 11:30:22.667277 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":525,"thread_start_us":415,"threads_started":1}
I20260810 11:30:22.678500 10484 raft_consensus.cc:1261] T 83e28eea13a14c64b9d3a4ed5a6b7ed2 P 2972efc435eb470c88b7c8689c4b96e1 [term 1 FOLLOWER]: Refusing update from remote peer 524610df19f84cfd88b4b7d4c3aaacdc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 11:30:22.678586 10346 raft_consensus.cc:1261] T 83e28eea13a14c64b9d3a4ed5a6b7ed2 P 600d04a3960143f89a8e1bee5bcd65e7 [term 1 FOLLOWER]: Refusing update from remote peer 524610df19f84cfd88b4b7d4c3aaacdc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 11:30:22.679397 10566 consensus_queue.cc:1035] T 83e28eea13a14c64b9d3a4ed5a6b7ed2 P 524610df19f84cfd88b4b7d4c3aaacdc [LEADER]: Connected to new peer: Peer: permanent_uuid: "2972efc435eb470c88b7c8689c4b96e1" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 38593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 11:30:22.679786 10561 consensus_queue.cc:1035] T 83e28eea13a14c64b9d3a4ed5a6b7ed2 P 524610df19f84cfd88b4b7d4c3aaacdc [LEADER]: Connected to new peer: Peer: permanent_uuid: "600d04a3960143f89a8e1bee5bcd65e7" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 45797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 11:30:22.724562 10552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:22.714856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.194:55023 (local address 127.8.18.193:35655)
0810 11:30:22.715085 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0810 11:30:22.715093 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:30:22.716285 (+ 1192us) server_negotiation.cc:407] Connection header received
0810 11:30:22.716622 (+ 337us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:30:22.716628 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:30:22.716700 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:30:22.716783 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 11:30:22.717549 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:22.718671 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:22.722205 (+ 3534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:22.722887 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:22.723070 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:22.723677 (+ 607us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 11:30:22.723788 (+ 111us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 11:30:22.723900 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0810 11:30:22.724148 (+ 248us) server_negotiation.cc:299] Negotiation successful
0810 11:30:22.724247 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260810 11:30:22.724730 10602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:22.714900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.193:35655 (local address 127.8.18.194:55023)
0810 11:30:22.716168 (+ 1268us) negotiation.cc:107] Waiting for socket to connect
0810 11:30:22.716191 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0810 11:30:22.716419 (+ 228us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:30:22.716836 (+ 417us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:30:22.716844 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:30:22.716863 (+ 19us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 11:30:22.717286 (+ 423us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:30:22.717299 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:30:22.718818 (+ 1519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:30:22.718825 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:30:22.721994 (+ 3169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:30:22.722014 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:30:22.723106 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:30:22.723113 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:30:22.723318 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:22.723401 (+ 83us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 11:30:22.723908 (+ 507us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 11:30:22.723922 (+ 14us) client_negotiation.cc:769] Sending connection context
0810 11:30:22.724094 (+ 172us) client_negotiation.cc:240] Negotiation successful
0810 11:30:22.724254 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1012,"thread_start_us":855,"threads_started":1}
I20260810 11:30:22.780354 10037 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41810:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 11:30:22.780704 10037 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41810:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 11:30:22.785389 10037 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ed0668521a7b4c0e82440311f75dd278: Sending DeleteTablet for 3 replicas of tablet 83e28eea13a14c64b9d3a4ed5a6b7ed2
I20260810 11:30:22.787256 10464 tablet_service.cc:1514] Processing DeleteTablet for tablet 83e28eea13a14c64b9d3a4ed5a6b7ed2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:30:22 UTC) from {username='kudu', principal='kudu/127.8.18.254@KRBTEST.COM'} at 127.0.0.1:40806
I20260810 11:30:22.787639 10179 tablet_service.cc:1514] Processing DeleteTablet for tablet 83e28eea13a14c64b9d3a4ed5a6b7ed2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:30:22 UTC) from {username='kudu', principal='kudu/127.8.18.254@KRBTEST.COM'} at 127.0.0.1:47350
I20260810 11:30:22.788935 10321 tablet_service.cc:1514] Processing DeleteTablet for tablet 83e28eea13a14c64b9d3a4ed5a6b7ed2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:30:22 UTC) from {username='kudu', principal='kudu/127.8.18.254@KRBTEST.COM'} at 127.0.0.1:47416
I20260810 11:30:22.790884 10037 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41810:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 11:30:22.792243 10037 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 11:30:22.805332 10320 tablet_service.cc:1467] Processing CreateTablet for tablet e93911a48ed244ab97338ff95f3b4074 (DEFAULT_TABLE table=test-table [id=688825a098954b3ca13ad5f6a2ecaffd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:30:22.806147 10320 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e93911a48ed244ab97338ff95f3b4074. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:30:22.809906 10606 tablet_replica.cc:330] T 83e28eea13a14c64b9d3a4ed5a6b7ed2 P 600d04a3960143f89a8e1bee5bcd65e7: stopping tablet replica
I20260810 11:30:22.809983 10178 tablet_service.cc:1467] Processing CreateTablet for tablet e93911a48ed244ab97338ff95f3b4074 (DEFAULT_TABLE table=test-table [id=688825a098954b3ca13ad5f6a2ecaffd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:30:22.810642 10178 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e93911a48ed244ab97338ff95f3b4074. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:30:22.810853 10606 raft_consensus.cc:2229] T 83e28eea13a14c64b9d3a4ed5a6b7ed2 P 600d04a3960143f89a8e1bee5bcd65e7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 11:30:22.811339 10606 raft_consensus.cc:2258] T 83e28eea13a14c64b9d3a4ed5a6b7ed2 P 600d04a3960143f89a8e1bee5bcd65e7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:30:22.813706 10606 ts_tablet_manager.cc:1905] T 83e28eea13a14c64b9d3a4ed5a6b7ed2 P 600d04a3960143f89a8e1bee5bcd65e7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:30:22.813977 10463 tablet_service.cc:1467] Processing CreateTablet for tablet e93911a48ed244ab97338ff95f3b4074 (DEFAULT_TABLE table=test-table [id=688825a098954b3ca13ad5f6a2ecaffd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:30:22.814535 10463 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e93911a48ed244ab97338ff95f3b4074. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:30:22.818487 10605 tablet_replica.cc:330] T 83e28eea13a14c64b9d3a4ed5a6b7ed2 P 2972efc435eb470c88b7c8689c4b96e1: stopping tablet replica
I20260810 11:30:22.819166 10605 raft_consensus.cc:2229] T 83e28eea13a14c64b9d3a4ed5a6b7ed2 P 2972efc435eb470c88b7c8689c4b96e1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 11:30:22.819685 10605 raft_consensus.cc:2258] T 83e28eea13a14c64b9d3a4ed5a6b7ed2 P 2972efc435eb470c88b7c8689c4b96e1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:30:22.822216 10605 ts_tablet_manager.cc:1905] T 83e28eea13a14c64b9d3a4ed5a6b7ed2 P 2972efc435eb470c88b7c8689c4b96e1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:30:22.827616 10606 ts_tablet_manager.cc:1918] T 83e28eea13a14c64b9d3a4ed5a6b7ed2 P 600d04a3960143f89a8e1bee5bcd65e7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 11:30:22.831816 10555 tablet_bootstrap.cc:492] T e93911a48ed244ab97338ff95f3b4074 P 2972efc435eb470c88b7c8689c4b96e1: Bootstrap starting.
I20260810 11:30:22.833835 10555 tablet_bootstrap.cc:654] T e93911a48ed244ab97338ff95f3b4074 P 2972efc435eb470c88b7c8689c4b96e1: Neither blocks nor log segments found. Creating new log.
I20260810 11:30:22.834229 10606 log.cc:1196] T 83e28eea13a14c64b9d3a4ed5a6b7ed2 P 600d04a3960143f89a8e1bee5bcd65e7: Deleting WAL directory at /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786361401922362-8267-0/minicluster-data/ts-1/wal/wals/83e28eea13a14c64b9d3a4ed5a6b7ed2
I20260810 11:30:22.835045 10606 ts_tablet_manager.cc:1939] T 83e28eea13a14c64b9d3a4ed5a6b7ed2 P 600d04a3960143f89a8e1bee5bcd65e7: Deleting consensus metadata
I20260810 11:30:22.838318 10556 tablet_bootstrap.cc:492] T e93911a48ed244ab97338ff95f3b4074 P 524610df19f84cfd88b4b7d4c3aaacdc: Bootstrap starting.
I20260810 11:30:22.840408 10556 tablet_bootstrap.cc:654] T e93911a48ed244ab97338ff95f3b4074 P 524610df19f84cfd88b4b7d4c3aaacdc: Neither blocks nor log segments found. Creating new log.
I20260810 11:30:22.840914 10023 catalog_manager.cc:4841] TS 600d04a3960143f89a8e1bee5bcd65e7 (127.8.18.194:45797): tablet 83e28eea13a14c64b9d3a4ed5a6b7ed2 (table test-table [id=f01dd88603ed4c97ac655489cded0178]) successfully deleted
I20260810 11:30:22.841645 10605 ts_tablet_manager.cc:1918] T 83e28eea13a14c64b9d3a4ed5a6b7ed2 P 2972efc435eb470c88b7c8689c4b96e1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 11:30:22.841960 10605 log.cc:1196] T 83e28eea13a14c64b9d3a4ed5a6b7ed2 P 2972efc435eb470c88b7c8689c4b96e1: Deleting WAL directory at /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786361401922362-8267-0/minicluster-data/ts-2/wal/wals/83e28eea13a14c64b9d3a4ed5a6b7ed2
I20260810 11:30:22.842694 10605 ts_tablet_manager.cc:1939] T 83e28eea13a14c64b9d3a4ed5a6b7ed2 P 2972efc435eb470c88b7c8689c4b96e1: Deleting consensus metadata
I20260810 11:30:22.845057 10024 catalog_manager.cc:4841] TS 2972efc435eb470c88b7c8689c4b96e1 (127.8.18.195:38593): tablet 83e28eea13a14c64b9d3a4ed5a6b7ed2 (table test-table [id=f01dd88603ed4c97ac655489cded0178]) successfully deleted
I20260810 11:30:22.848546 10557 tablet_bootstrap.cc:492] T e93911a48ed244ab97338ff95f3b4074 P 600d04a3960143f89a8e1bee5bcd65e7: Bootstrap starting.
I20260810 11:30:22.849858 10607 tablet_replica.cc:330] T 83e28eea13a14c64b9d3a4ed5a6b7ed2 P 524610df19f84cfd88b4b7d4c3aaacdc: stopping tablet replica
I20260810 11:30:22.850448 10557 tablet_bootstrap.cc:654] T e93911a48ed244ab97338ff95f3b4074 P 600d04a3960143f89a8e1bee5bcd65e7: Neither blocks nor log segments found. Creating new log.
I20260810 11:30:22.850449 10607 raft_consensus.cc:2229] T 83e28eea13a14c64b9d3a4ed5a6b7ed2 P 524610df19f84cfd88b4b7d4c3aaacdc [term 1 LEADER]: Raft consensus shutting down.
I20260810 11:30:22.851332 10607 raft_consensus.cc:2258] T 83e28eea13a14c64b9d3a4ed5a6b7ed2 P 524610df19f84cfd88b4b7d4c3aaacdc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:30:22.853102 10556 tablet_bootstrap.cc:492] T e93911a48ed244ab97338ff95f3b4074 P 524610df19f84cfd88b4b7d4c3aaacdc: No bootstrap required, opened a new log
I20260810 11:30:22.853372 10556 ts_tablet_manager.cc:1397] T e93911a48ed244ab97338ff95f3b4074 P 524610df19f84cfd88b4b7d4c3aaacdc: Time spent bootstrapping tablet: real 0.015s user 0.001s sys 0.006s
I20260810 11:30:22.855406 10556 raft_consensus.cc:348] T e93911a48ed244ab97338ff95f3b4074 P 524610df19f84cfd88b4b7d4c3aaacdc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "524610df19f84cfd88b4b7d4c3aaacdc" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 35655 } } peers { permanent_uuid: "600d04a3960143f89a8e1bee5bcd65e7" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 45797 } } peers { permanent_uuid: "2972efc435eb470c88b7c8689c4b96e1" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 38593 } }
I20260810 11:30:22.856401 10556 raft_consensus.cc:374] T e93911a48ed244ab97338ff95f3b4074 P 524610df19f84cfd88b4b7d4c3aaacdc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:30:22.857582 10556 raft_consensus.cc:729] T e93911a48ed244ab97338ff95f3b4074 P 524610df19f84cfd88b4b7d4c3aaacdc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 524610df19f84cfd88b4b7d4c3aaacdc, State: Initialized, Role: FOLLOWER
I20260810 11:30:22.858057 10556 consensus_queue.cc:260] T e93911a48ed244ab97338ff95f3b4074 P 524610df19f84cfd88b4b7d4c3aaacdc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "524610df19f84cfd88b4b7d4c3aaacdc" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 35655 } } peers { permanent_uuid: "600d04a3960143f89a8e1bee5bcd65e7" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 45797 } } peers { permanent_uuid: "2972efc435eb470c88b7c8689c4b96e1" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 38593 } }
I20260810 11:30:22.858554 10555 tablet_bootstrap.cc:492] T e93911a48ed244ab97338ff95f3b4074 P 2972efc435eb470c88b7c8689c4b96e1: No bootstrap required, opened a new log
I20260810 11:30:22.859313 10556 ts_tablet_manager.cc:1428] T e93911a48ed244ab97338ff95f3b4074 P 524610df19f84cfd88b4b7d4c3aaacdc: Time spent starting tablet: real 0.006s user 0.000s sys 0.002s
I20260810 11:30:22.859587 10555 ts_tablet_manager.cc:1397] T e93911a48ed244ab97338ff95f3b4074 P 2972efc435eb470c88b7c8689c4b96e1: Time spent bootstrapping tablet: real 0.028s user 0.009s sys 0.001s
I20260810 11:30:22.860605 10555 raft_consensus.cc:348] T e93911a48ed244ab97338ff95f3b4074 P 2972efc435eb470c88b7c8689c4b96e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "524610df19f84cfd88b4b7d4c3aaacdc" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 35655 } } peers { permanent_uuid: "600d04a3960143f89a8e1bee5bcd65e7" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 45797 } } peers { permanent_uuid: "2972efc435eb470c88b7c8689c4b96e1" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 38593 } }
I20260810 11:30:22.861171 10555 raft_consensus.cc:374] T e93911a48ed244ab97338ff95f3b4074 P 2972efc435eb470c88b7c8689c4b96e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:30:22.861323 10555 raft_consensus.cc:729] T e93911a48ed244ab97338ff95f3b4074 P 2972efc435eb470c88b7c8689c4b96e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2972efc435eb470c88b7c8689c4b96e1, State: Initialized, Role: FOLLOWER
I20260810 11:30:22.857045 10607 ts_tablet_manager.cc:1905] T 83e28eea13a14c64b9d3a4ed5a6b7ed2 P 524610df19f84cfd88b4b7d4c3aaacdc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:30:22.861810 10555 consensus_queue.cc:260] T e93911a48ed244ab97338ff95f3b4074 P 2972efc435eb470c88b7c8689c4b96e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "524610df19f84cfd88b4b7d4c3aaacdc" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 35655 } } peers { permanent_uuid: "600d04a3960143f89a8e1bee5bcd65e7" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 45797 } } peers { permanent_uuid: "2972efc435eb470c88b7c8689c4b96e1" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 38593 } }
I20260810 11:30:22.862485 10555 ts_tablet_manager.cc:1428] T e93911a48ed244ab97338ff95f3b4074 P 2972efc435eb470c88b7c8689c4b96e1: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260810 11:30:22.871487 10607 ts_tablet_manager.cc:1918] T 83e28eea13a14c64b9d3a4ed5a6b7ed2 P 524610df19f84cfd88b4b7d4c3aaacdc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 11:30:22.871711 10607 log.cc:1196] T 83e28eea13a14c64b9d3a4ed5a6b7ed2 P 524610df19f84cfd88b4b7d4c3aaacdc: Deleting WAL directory at /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786361401922362-8267-0/minicluster-data/ts-0/wal/wals/83e28eea13a14c64b9d3a4ed5a6b7ed2
I20260810 11:30:22.871731 10557 tablet_bootstrap.cc:492] T e93911a48ed244ab97338ff95f3b4074 P 600d04a3960143f89a8e1bee5bcd65e7: No bootstrap required, opened a new log
I20260810 11:30:22.871930 10557 ts_tablet_manager.cc:1397] T e93911a48ed244ab97338ff95f3b4074 P 600d04a3960143f89a8e1bee5bcd65e7: Time spent bootstrapping tablet: real 0.023s user 0.007s sys 0.003s
I20260810 11:30:22.872361 10607 ts_tablet_manager.cc:1939] T 83e28eea13a14c64b9d3a4ed5a6b7ed2 P 524610df19f84cfd88b4b7d4c3aaacdc: Deleting consensus metadata
I20260810 11:30:22.872860 10557 raft_consensus.cc:348] T e93911a48ed244ab97338ff95f3b4074 P 600d04a3960143f89a8e1bee5bcd65e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "524610df19f84cfd88b4b7d4c3aaacdc" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 35655 } } peers { permanent_uuid: "600d04a3960143f89a8e1bee5bcd65e7" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 45797 } } peers { permanent_uuid: "2972efc435eb470c88b7c8689c4b96e1" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 38593 } }
I20260810 11:30:22.873144 10557 raft_consensus.cc:374] T e93911a48ed244ab97338ff95f3b4074 P 600d04a3960143f89a8e1bee5bcd65e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:30:22.873242 10557 raft_consensus.cc:729] T e93911a48ed244ab97338ff95f3b4074 P 600d04a3960143f89a8e1bee5bcd65e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 600d04a3960143f89a8e1bee5bcd65e7, State: Initialized, Role: FOLLOWER
I20260810 11:30:22.873530 10557 consensus_queue.cc:260] T e93911a48ed244ab97338ff95f3b4074 P 600d04a3960143f89a8e1bee5bcd65e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "524610df19f84cfd88b4b7d4c3aaacdc" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 35655 } } peers { permanent_uuid: "600d04a3960143f89a8e1bee5bcd65e7" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 45797 } } peers { permanent_uuid: "2972efc435eb470c88b7c8689c4b96e1" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 38593 } }
I20260810 11:30:22.874626 10022 catalog_manager.cc:4841] TS 524610df19f84cfd88b4b7d4c3aaacdc (127.8.18.193:35655): tablet 83e28eea13a14c64b9d3a4ed5a6b7ed2 (table test-table [id=f01dd88603ed4c97ac655489cded0178]) successfully deleted
I20260810 11:30:22.875488 10557 ts_tablet_manager.cc:1428] T e93911a48ed244ab97338ff95f3b4074 P 600d04a3960143f89a8e1bee5bcd65e7: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 11:30:23.021502 10600 raft_consensus.cc:482] T e93911a48ed244ab97338ff95f3b4074 P 600d04a3960143f89a8e1bee5bcd65e7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 11:30:23.021767 10600 raft_consensus.cc:504] T e93911a48ed244ab97338ff95f3b4074 P 600d04a3960143f89a8e1bee5bcd65e7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "524610df19f84cfd88b4b7d4c3aaacdc" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 35655 } } peers { permanent_uuid: "600d04a3960143f89a8e1bee5bcd65e7" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 45797 } } peers { permanent_uuid: "2972efc435eb470c88b7c8689c4b96e1" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 38593 } }
I20260810 11:30:23.022781 10600 leader_election.cc:288] T e93911a48ed244ab97338ff95f3b4074 P 600d04a3960143f89a8e1bee5bcd65e7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 524610df19f84cfd88b4b7d4c3aaacdc (127.8.18.193:35655), 2972efc435eb470c88b7c8689c4b96e1 (127.8.18.195:38593)
I20260810 11:30:23.023550 10199 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e93911a48ed244ab97338ff95f3b4074" candidate_uuid: "600d04a3960143f89a8e1bee5bcd65e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "524610df19f84cfd88b4b7d4c3aaacdc" is_pre_election: true
I20260810 11:30:23.023653 10483 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e93911a48ed244ab97338ff95f3b4074" candidate_uuid: "600d04a3960143f89a8e1bee5bcd65e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2972efc435eb470c88b7c8689c4b96e1" is_pre_election: true
I20260810 11:30:23.023838 10199 raft_consensus.cc:2454] T e93911a48ed244ab97338ff95f3b4074 P 524610df19f84cfd88b4b7d4c3aaacdc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 600d04a3960143f89a8e1bee5bcd65e7 in term 0.
I20260810 11:30:23.023952 10483 raft_consensus.cc:2454] T e93911a48ed244ab97338ff95f3b4074 P 2972efc435eb470c88b7c8689c4b96e1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 600d04a3960143f89a8e1bee5bcd65e7 in term 0.
I20260810 11:30:23.024577 10275 leader_election.cc:302] T e93911a48ed244ab97338ff95f3b4074 P 600d04a3960143f89a8e1bee5bcd65e7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2972efc435eb470c88b7c8689c4b96e1, 600d04a3960143f89a8e1bee5bcd65e7; no voters:
I20260810 11:30:23.024982 10600 raft_consensus.cc:2781] T e93911a48ed244ab97338ff95f3b4074 P 600d04a3960143f89a8e1bee5bcd65e7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 11:30:23.025127 10600 raft_consensus.cc:482] T e93911a48ed244ab97338ff95f3b4074 P 600d04a3960143f89a8e1bee5bcd65e7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 11:30:23.025246 10600 raft_consensus.cc:3037] T e93911a48ed244ab97338ff95f3b4074 P 600d04a3960143f89a8e1bee5bcd65e7 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:30:23.028988 10600 raft_consensus.cc:504] T e93911a48ed244ab97338ff95f3b4074 P 600d04a3960143f89a8e1bee5bcd65e7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "524610df19f84cfd88b4b7d4c3aaacdc" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 35655 } } peers { permanent_uuid: "600d04a3960143f89a8e1bee5bcd65e7" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 45797 } } peers { permanent_uuid: "2972efc435eb470c88b7c8689c4b96e1" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 38593 } }
I20260810 11:30:23.030702 10199 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e93911a48ed244ab97338ff95f3b4074" candidate_uuid: "600d04a3960143f89a8e1bee5bcd65e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "524610df19f84cfd88b4b7d4c3aaacdc"
I20260810 11:30:23.030745 10483 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e93911a48ed244ab97338ff95f3b4074" candidate_uuid: "600d04a3960143f89a8e1bee5bcd65e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2972efc435eb470c88b7c8689c4b96e1"
I20260810 11:30:23.030997 10199 raft_consensus.cc:3037] T e93911a48ed244ab97338ff95f3b4074 P 524610df19f84cfd88b4b7d4c3aaacdc [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:30:23.031096 10483 raft_consensus.cc:3037] T e93911a48ed244ab97338ff95f3b4074 P 2972efc435eb470c88b7c8689c4b96e1 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:30:23.035185 10483 raft_consensus.cc:2454] T e93911a48ed244ab97338ff95f3b4074 P 2972efc435eb470c88b7c8689c4b96e1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 600d04a3960143f89a8e1bee5bcd65e7 in term 1.
I20260810 11:30:23.035184 10199 raft_consensus.cc:2454] T e93911a48ed244ab97338ff95f3b4074 P 524610df19f84cfd88b4b7d4c3aaacdc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 600d04a3960143f89a8e1bee5bcd65e7 in term 1.
I20260810 11:30:23.035929 10273 leader_election.cc:302] T e93911a48ed244ab97338ff95f3b4074 P 600d04a3960143f89a8e1bee5bcd65e7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 524610df19f84cfd88b4b7d4c3aaacdc, 600d04a3960143f89a8e1bee5bcd65e7; no voters:
I20260810 11:30:23.036036 10600 leader_election.cc:288] T e93911a48ed244ab97338ff95f3b4074 P 600d04a3960143f89a8e1bee5bcd65e7 [CANDIDATE]: Term 1 election: Requested vote from peers 524610df19f84cfd88b4b7d4c3aaacdc (127.8.18.193:35655), 2972efc435eb470c88b7c8689c4b96e1 (127.8.18.195:38593)
I20260810 11:30:23.036398 10563 raft_consensus.cc:2781] T e93911a48ed244ab97338ff95f3b4074 P 600d04a3960143f89a8e1bee5bcd65e7 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:30:23.036581 10563 raft_consensus.cc:686] T e93911a48ed244ab97338ff95f3b4074 P 600d04a3960143f89a8e1bee5bcd65e7 [term 1 LEADER]: Becoming Leader. State: Replica: 600d04a3960143f89a8e1bee5bcd65e7, State: Running, Role: LEADER
I20260810 11:30:23.036906 10563 consensus_queue.cc:237] T e93911a48ed244ab97338ff95f3b4074 P 600d04a3960143f89a8e1bee5bcd65e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "524610df19f84cfd88b4b7d4c3aaacdc" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 35655 } } peers { permanent_uuid: "600d04a3960143f89a8e1bee5bcd65e7" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 45797 } } peers { permanent_uuid: "2972efc435eb470c88b7c8689c4b96e1" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 38593 } }
I20260810 11:30:23.042344 10037 catalog_manager.cc:5495] T e93911a48ed244ab97338ff95f3b4074 P 600d04a3960143f89a8e1bee5bcd65e7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 600d04a3960143f89a8e1bee5bcd65e7 (127.8.18.194). New cstate: current_term: 1 leader_uuid: "600d04a3960143f89a8e1bee5bcd65e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "524610df19f84cfd88b4b7d4c3aaacdc" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 35655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "600d04a3960143f89a8e1bee5bcd65e7" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 45797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2972efc435eb470c88b7c8689c4b96e1" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 38593 } health_report { overall_health: UNKNOWN } } }
I20260810 11:30:23.087563 10554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:23.079176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47422 (local address 127.8.18.194:45797)
0810 11:30:23.079422 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0810 11:30:23.079431 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 11:30:23.079623 (+ 192us) server_negotiation.cc:407] Connection header received
0810 11:30:23.079847 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:30:23.079852 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:30:23.079953 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:30:23.080080 (+ 127us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 11:30:23.080581 (+ 501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:23.081861 (+ 1280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:23.084154 (+ 2293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:23.084883 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:23.085048 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:23.086052 (+ 1004us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 11:30:23.086286 (+ 234us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 11:30:23.086403 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0810 11:30:23.086982 (+ 579us) server_negotiation.cc:299] Negotiation successful
0810 11:30:23.087089 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260810 11:30:23.108376 10602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:23.100342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.194:45797 (local address 127.8.18.194:42857)
0810 11:30:23.100678 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0810 11:30:23.100702 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0810 11:30:23.100831 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:30:23.101556 (+ 725us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:30:23.101579 (+ 23us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:30:23.101600 (+ 21us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 11:30:23.101857 (+ 257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:30:23.101873 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:30:23.103714 (+ 1841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:30:23.103720 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:30:23.105508 (+ 1788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:30:23.105528 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:30:23.106926 (+ 1398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:30:23.106932 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:30:23.107120 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:23.107170 (+ 50us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 11:30:23.107751 (+ 581us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 11:30:23.107764 (+ 13us) client_negotiation.cc:769] Sending connection context
0810 11:30:23.107847 (+ 83us) client_negotiation.cc:240] Negotiation successful
0810 11:30:23.107990 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174}
I20260810 11:30:23.108403 10554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:23.100573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.194:42857 (local address 127.8.18.194:45797)
0810 11:30:23.100837 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0810 11:30:23.100844 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:30:23.100869 (+ 25us) server_negotiation.cc:407] Connection header received
0810 11:30:23.100947 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:30:23.100954 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:30:23.101122 (+ 168us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:30:23.101371 (+ 249us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 11:30:23.102055 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:23.103503 (+ 1448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:23.105716 (+ 2213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:23.106694 (+ 978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:23.106843 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:23.107438 (+ 595us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 11:30:23.107622 (+ 184us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 11:30:23.107724 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0810 11:30:23.107902 (+ 178us) server_negotiation.cc:299] Negotiation successful
0810 11:30:23.108002 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260810 11:30:23.121330 10483 raft_consensus.cc:1261] T e93911a48ed244ab97338ff95f3b4074 P 2972efc435eb470c88b7c8689c4b96e1 [term 1 FOLLOWER]: Refusing update from remote peer 600d04a3960143f89a8e1bee5bcd65e7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 11:30:23.122503 10199 raft_consensus.cc:1261] T e93911a48ed244ab97338ff95f3b4074 P 524610df19f84cfd88b4b7d4c3aaacdc [term 1 FOLLOWER]: Refusing update from remote peer 600d04a3960143f89a8e1bee5bcd65e7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 11:30:23.123546 10600 consensus_queue.cc:1035] T e93911a48ed244ab97338ff95f3b4074 P 600d04a3960143f89a8e1bee5bcd65e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "524610df19f84cfd88b4b7d4c3aaacdc" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 35655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 11:30:23.124931 10600 consensus_queue.cc:1035] T e93911a48ed244ab97338ff95f3b4074 P 600d04a3960143f89a8e1bee5bcd65e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2972efc435eb470c88b7c8689c4b96e1" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 38593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 11:30:23.153282 10586 mvcc.cc:205] Tried to move back new op lower bound from 7316936389098467328 to 7316936388762730496. Current Snapshot: MvccSnapshot[applied={T|T < 7316936389098467328}]
W20260810 11:30:23.172952 10584 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260810 11:30:24.837852 10037 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41810:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 11:30:24.838377 10037 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41810:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 11:30:24.848742 10037 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ed0668521a7b4c0e82440311f75dd278: Sending DeleteTablet for 3 replicas of tablet e93911a48ed244ab97338ff95f3b4074
I20260810 11:30:24.850497 10178 tablet_service.cc:1514] Processing DeleteTablet for tablet e93911a48ed244ab97338ff95f3b4074 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:30:24 UTC) from {username='kudu', principal='kudu/127.8.18.254@KRBTEST.COM'} at 127.0.0.1:47350
I20260810 11:30:24.850909 10320 tablet_service.cc:1514] Processing DeleteTablet for tablet e93911a48ed244ab97338ff95f3b4074 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:30:24 UTC) from {username='kudu', principal='kudu/127.8.18.254@KRBTEST.COM'} at 127.0.0.1:47416
I20260810 11:30:24.850952 10464 tablet_service.cc:1514] Processing DeleteTablet for tablet e93911a48ed244ab97338ff95f3b4074 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:30:24 UTC) from {username='kudu', principal='kudu/127.8.18.254@KRBTEST.COM'} at 127.0.0.1:40806
I20260810 11:30:24.863950 10635 tablet_replica.cc:330] T e93911a48ed244ab97338ff95f3b4074 P 600d04a3960143f89a8e1bee5bcd65e7: stopping tablet replica
I20260810 11:30:24.864686 10635 raft_consensus.cc:2229] T e93911a48ed244ab97338ff95f3b4074 P 600d04a3960143f89a8e1bee5bcd65e7 [term 1 LEADER]: Raft consensus shutting down.
I20260810 11:30:24.866308 10635 raft_consensus.cc:2258] T e93911a48ed244ab97338ff95f3b4074 P 600d04a3960143f89a8e1bee5bcd65e7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:30:24.868072 10634 tablet_replica.cc:330] T e93911a48ed244ab97338ff95f3b4074 P 524610df19f84cfd88b4b7d4c3aaacdc: stopping tablet replica
I20260810 11:30:24.868073 10636 tablet_replica.cc:330] T e93911a48ed244ab97338ff95f3b4074 P 2972efc435eb470c88b7c8689c4b96e1: stopping tablet replica
I20260810 11:30:24.868507 10636 raft_consensus.cc:2229] T e93911a48ed244ab97338ff95f3b4074 P 2972efc435eb470c88b7c8689c4b96e1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 11:30:24.868588 10634 raft_consensus.cc:2229] T e93911a48ed244ab97338ff95f3b4074 P 524610df19f84cfd88b4b7d4c3aaacdc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 11:30:24.869194 10635 ts_tablet_manager.cc:1905] T e93911a48ed244ab97338ff95f3b4074 P 600d04a3960143f89a8e1bee5bcd65e7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:30:24.869254 10634 raft_consensus.cc:2258] T e93911a48ed244ab97338ff95f3b4074 P 524610df19f84cfd88b4b7d4c3aaacdc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:30:24.869355 10636 raft_consensus.cc:2258] T e93911a48ed244ab97338ff95f3b4074 P 2972efc435eb470c88b7c8689c4b96e1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:30:24.871224 10636 ts_tablet_manager.cc:1905] T e93911a48ed244ab97338ff95f3b4074 P 2972efc435eb470c88b7c8689c4b96e1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:30:24.871315 10634 ts_tablet_manager.cc:1905] T e93911a48ed244ab97338ff95f3b4074 P 524610df19f84cfd88b4b7d4c3aaacdc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:30:24.883194 10635 ts_tablet_manager.cc:1918] T e93911a48ed244ab97338ff95f3b4074 P 600d04a3960143f89a8e1bee5bcd65e7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260810 11:30:24.883687 10635 log.cc:1196] T e93911a48ed244ab97338ff95f3b4074 P 600d04a3960143f89a8e1bee5bcd65e7: Deleting WAL directory at /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786361401922362-8267-0/minicluster-data/ts-1/wal/wals/e93911a48ed244ab97338ff95f3b4074
I20260810 11:30:24.885151 10635 ts_tablet_manager.cc:1939] T e93911a48ed244ab97338ff95f3b4074 P 600d04a3960143f89a8e1bee5bcd65e7: Deleting consensus metadata
I20260810 11:30:24.886071 10636 ts_tablet_manager.cc:1918] T e93911a48ed244ab97338ff95f3b4074 P 2972efc435eb470c88b7c8689c4b96e1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260810 11:30:24.886132 10634 ts_tablet_manager.cc:1918] T e93911a48ed244ab97338ff95f3b4074 P 524610df19f84cfd88b4b7d4c3aaacdc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260810 11:30:24.886379 10634 log.cc:1196] T e93911a48ed244ab97338ff95f3b4074 P 524610df19f84cfd88b4b7d4c3aaacdc: Deleting WAL directory at /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786361401922362-8267-0/minicluster-data/ts-0/wal/wals/e93911a48ed244ab97338ff95f3b4074
I20260810 11:30:24.886552 10636 log.cc:1196] T e93911a48ed244ab97338ff95f3b4074 P 2972efc435eb470c88b7c8689c4b96e1: Deleting WAL directory at /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786361401922362-8267-0/minicluster-data/ts-2/wal/wals/e93911a48ed244ab97338ff95f3b4074
I20260810 11:30:24.887511 10634 ts_tablet_manager.cc:1939] T e93911a48ed244ab97338ff95f3b4074 P 524610df19f84cfd88b4b7d4c3aaacdc: Deleting consensus metadata
I20260810 11:30:24.887921 10023 catalog_manager.cc:4841] TS 600d04a3960143f89a8e1bee5bcd65e7 (127.8.18.194:45797): tablet e93911a48ed244ab97338ff95f3b4074 (table test-table [id=688825a098954b3ca13ad5f6a2ecaffd]) successfully deleted
I20260810 11:30:24.888432 10636 ts_tablet_manager.cc:1939] T e93911a48ed244ab97338ff95f3b4074 P 2972efc435eb470c88b7c8689c4b96e1: Deleting consensus metadata
I20260810 11:30:24.890024 10022 catalog_manager.cc:4841] TS 524610df19f84cfd88b4b7d4c3aaacdc (127.8.18.193:35655): tablet e93911a48ed244ab97338ff95f3b4074 (table test-table [id=688825a098954b3ca13ad5f6a2ecaffd]) successfully deleted
I20260810 11:30:24.890430 10024 catalog_manager.cc:4841] TS 2972efc435eb470c88b7c8689c4b96e1 (127.8.18.195:38593): tablet e93911a48ed244ab97338ff95f3b4074 (table test-table [id=688825a098954b3ca13ad5f6a2ecaffd]) successfully deleted
Aug 10 11:30:24 dist-test-slave-srgv krb5kdc[9992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361424, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 10 11:30:24 dist-test-slave-srgv krb5kdc[9992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361424, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.8.18.254@KRBTEST.COM
I20260810 11:30:24.971127 10643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:24.934366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41838 (local address 127.8.18.254:40395)
0810 11:30:24.935942 (+ 1576us) server_negotiation.cc:206] Beginning negotiation
0810 11:30:24.935963 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0810 11:30:24.936024 (+ 61us) server_negotiation.cc:407] Connection header received
0810 11:30:24.936408 (+ 384us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:30:24.936414 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:30:24.936528 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:30:24.936726 (+ 198us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:30:24.939738 (+ 3012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:24.941283 (+ 1545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:24.944071 (+ 2788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:24.944910 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:24.945236 (+ 326us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:24.958598 (+ 13362us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:30:24.958614 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:30:24.958617 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:30:24.958704 (+ 87us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:30:24.964503 (+ 5799us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:24.966210 (+ 1707us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:24.966231 (+ 21us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:24.966236 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:24.966520 (+ 284us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:24.967310 (+ 790us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:24.967324 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:24.967326 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:24.968205 (+ 879us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:30:24.968767 (+ 562us) server_negotiation.cc:1035] Waiting for connection context
0810 11:30:24.969735 (+ 968us) server_negotiation.cc:299] Negotiation successful
0810 11:30:24.970279 (+ 544us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1264,"thread_start_us":865,"threads_started":1}
I20260810 11:30:24.979394 10037 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41838:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 11:30:24.980772 10037 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 11:30:24.994782 10178 tablet_service.cc:1467] Processing CreateTablet for tablet 346a41a691de42858d1a67ab7f25df4c (DEFAULT_TABLE table=test-table [id=388e1dc1095b4358b5ca1f1b0a98b379]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:30:24.995517 10320 tablet_service.cc:1467] Processing CreateTablet for tablet 346a41a691de42858d1a67ab7f25df4c (DEFAULT_TABLE table=test-table [id=388e1dc1095b4358b5ca1f1b0a98b379]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:30:24.995692 10178 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 346a41a691de42858d1a67ab7f25df4c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:30:24.995757 10464 tablet_service.cc:1467] Processing CreateTablet for tablet 346a41a691de42858d1a67ab7f25df4c (DEFAULT_TABLE table=test-table [id=388e1dc1095b4358b5ca1f1b0a98b379]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:30:24.996443 10320 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 346a41a691de42858d1a67ab7f25df4c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:30:24.996606 10464 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 346a41a691de42858d1a67ab7f25df4c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:30:25.007810 10645 tablet_bootstrap.cc:492] T 346a41a691de42858d1a67ab7f25df4c P 600d04a3960143f89a8e1bee5bcd65e7: Bootstrap starting.
I20260810 11:30:25.011479 10647 tablet_bootstrap.cc:492] T 346a41a691de42858d1a67ab7f25df4c P 524610df19f84cfd88b4b7d4c3aaacdc: Bootstrap starting.
I20260810 11:30:25.012574 10645 tablet_bootstrap.cc:654] T 346a41a691de42858d1a67ab7f25df4c P 600d04a3960143f89a8e1bee5bcd65e7: Neither blocks nor log segments found. Creating new log.
I20260810 11:30:25.019013 10648 tablet_bootstrap.cc:492] T 346a41a691de42858d1a67ab7f25df4c P 2972efc435eb470c88b7c8689c4b96e1: Bootstrap starting.
I20260810 11:30:25.020124 10647 tablet_bootstrap.cc:654] T 346a41a691de42858d1a67ab7f25df4c P 524610df19f84cfd88b4b7d4c3aaacdc: Neither blocks nor log segments found. Creating new log.
I20260810 11:30:25.021457 10645 tablet_bootstrap.cc:492] T 346a41a691de42858d1a67ab7f25df4c P 600d04a3960143f89a8e1bee5bcd65e7: No bootstrap required, opened a new log
I20260810 11:30:25.021905 10645 ts_tablet_manager.cc:1397] T 346a41a691de42858d1a67ab7f25df4c P 600d04a3960143f89a8e1bee5bcd65e7: Time spent bootstrapping tablet: real 0.014s user 0.000s sys 0.011s
I20260810 11:30:25.024608 10645 raft_consensus.cc:348] T 346a41a691de42858d1a67ab7f25df4c P 600d04a3960143f89a8e1bee5bcd65e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "524610df19f84cfd88b4b7d4c3aaacdc" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 35655 } } peers { permanent_uuid: "600d04a3960143f89a8e1bee5bcd65e7" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 45797 } } peers { permanent_uuid: "2972efc435eb470c88b7c8689c4b96e1" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 38593 } }
I20260810 11:30:25.025280 10645 raft_consensus.cc:374] T 346a41a691de42858d1a67ab7f25df4c P 600d04a3960143f89a8e1bee5bcd65e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:30:25.025771 10645 raft_consensus.cc:729] T 346a41a691de42858d1a67ab7f25df4c P 600d04a3960143f89a8e1bee5bcd65e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 600d04a3960143f89a8e1bee5bcd65e7, State: Initialized, Role: FOLLOWER
I20260810 11:30:25.026943 10648 tablet_bootstrap.cc:654] T 346a41a691de42858d1a67ab7f25df4c P 2972efc435eb470c88b7c8689c4b96e1: Neither blocks nor log segments found. Creating new log.
I20260810 11:30:25.026942 10645 consensus_queue.cc:260] T 346a41a691de42858d1a67ab7f25df4c P 600d04a3960143f89a8e1bee5bcd65e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "524610df19f84cfd88b4b7d4c3aaacdc" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 35655 } } peers { permanent_uuid: "600d04a3960143f89a8e1bee5bcd65e7" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 45797 } } peers { permanent_uuid: "2972efc435eb470c88b7c8689c4b96e1" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 38593 } }
I20260810 11:30:25.028132 10647 tablet_bootstrap.cc:492] T 346a41a691de42858d1a67ab7f25df4c P 524610df19f84cfd88b4b7d4c3aaacdc: No bootstrap required, opened a new log
I20260810 11:30:25.028523 10647 ts_tablet_manager.cc:1397] T 346a41a691de42858d1a67ab7f25df4c P 524610df19f84cfd88b4b7d4c3aaacdc: Time spent bootstrapping tablet: real 0.017s user 0.010s sys 0.005s
I20260810 11:30:25.031806 10647 raft_consensus.cc:348] T 346a41a691de42858d1a67ab7f25df4c P 524610df19f84cfd88b4b7d4c3aaacdc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "524610df19f84cfd88b4b7d4c3aaacdc" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 35655 } } peers { permanent_uuid: "600d04a3960143f89a8e1bee5bcd65e7" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 45797 } } peers { permanent_uuid: "2972efc435eb470c88b7c8689c4b96e1" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 38593 } }
I20260810 11:30:25.032352 10645 ts_tablet_manager.cc:1428] T 346a41a691de42858d1a67ab7f25df4c P 600d04a3960143f89a8e1bee5bcd65e7: Time spent starting tablet: real 0.010s user 0.000s sys 0.007s
I20260810 11:30:25.032362 10647 raft_consensus.cc:374] T 346a41a691de42858d1a67ab7f25df4c P 524610df19f84cfd88b4b7d4c3aaacdc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:30:25.032600 10647 raft_consensus.cc:729] T 346a41a691de42858d1a67ab7f25df4c P 524610df19f84cfd88b4b7d4c3aaacdc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 524610df19f84cfd88b4b7d4c3aaacdc, State: Initialized, Role: FOLLOWER
I20260810 11:30:25.033126 10647 consensus_queue.cc:260] T 346a41a691de42858d1a67ab7f25df4c P 524610df19f84cfd88b4b7d4c3aaacdc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "524610df19f84cfd88b4b7d4c3aaacdc" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 35655 } } peers { permanent_uuid: "600d04a3960143f89a8e1bee5bcd65e7" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 45797 } } peers { permanent_uuid: "2972efc435eb470c88b7c8689c4b96e1" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 38593 } }
I20260810 11:30:25.034781 10648 tablet_bootstrap.cc:492] T 346a41a691de42858d1a67ab7f25df4c P 2972efc435eb470c88b7c8689c4b96e1: No bootstrap required, opened a new log
I20260810 11:30:25.035089 10648 ts_tablet_manager.cc:1397] T 346a41a691de42858d1a67ab7f25df4c P 2972efc435eb470c88b7c8689c4b96e1: Time spent bootstrapping tablet: real 0.017s user 0.005s sys 0.007s
I20260810 11:30:25.037235 10648 raft_consensus.cc:348] T 346a41a691de42858d1a67ab7f25df4c P 2972efc435eb470c88b7c8689c4b96e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "524610df19f84cfd88b4b7d4c3aaacdc" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 35655 } } peers { permanent_uuid: "600d04a3960143f89a8e1bee5bcd65e7" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 45797 } } peers { permanent_uuid: "2972efc435eb470c88b7c8689c4b96e1" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 38593 } }
I20260810 11:30:25.037679 10647 ts_tablet_manager.cc:1428] T 346a41a691de42858d1a67ab7f25df4c P 524610df19f84cfd88b4b7d4c3aaacdc: Time spent starting tablet: real 0.009s user 0.008s sys 0.001s
I20260810 11:30:25.037776 10648 raft_consensus.cc:374] T 346a41a691de42858d1a67ab7f25df4c P 2972efc435eb470c88b7c8689c4b96e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:30:25.037957 10648 raft_consensus.cc:729] T 346a41a691de42858d1a67ab7f25df4c P 2972efc435eb470c88b7c8689c4b96e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2972efc435eb470c88b7c8689c4b96e1, State: Initialized, Role: FOLLOWER
I20260810 11:30:25.038484 10648 consensus_queue.cc:260] T 346a41a691de42858d1a67ab7f25df4c P 2972efc435eb470c88b7c8689c4b96e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "524610df19f84cfd88b4b7d4c3aaacdc" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 35655 } } peers { permanent_uuid: "600d04a3960143f89a8e1bee5bcd65e7" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 45797 } } peers { permanent_uuid: "2972efc435eb470c88b7c8689c4b96e1" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 38593 } }
I20260810 11:30:25.040604 10648 ts_tablet_manager.cc:1428] T 346a41a691de42858d1a67ab7f25df4c P 2972efc435eb470c88b7c8689c4b96e1: Time spent starting tablet: real 0.005s user 0.003s sys 0.003s
I20260810 11:30:25.057767 10653 raft_consensus.cc:482] T 346a41a691de42858d1a67ab7f25df4c P 524610df19f84cfd88b4b7d4c3aaacdc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 11:30:25.058306 10653 raft_consensus.cc:504] T 346a41a691de42858d1a67ab7f25df4c P 524610df19f84cfd88b4b7d4c3aaacdc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "524610df19f84cfd88b4b7d4c3aaacdc" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 35655 } } peers { permanent_uuid: "600d04a3960143f89a8e1bee5bcd65e7" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 45797 } } peers { permanent_uuid: "2972efc435eb470c88b7c8689c4b96e1" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 38593 } }
I20260810 11:30:25.059729 10653 leader_election.cc:288] T 346a41a691de42858d1a67ab7f25df4c P 524610df19f84cfd88b4b7d4c3aaacdc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 600d04a3960143f89a8e1bee5bcd65e7 (127.8.18.194:45797), 2972efc435eb470c88b7c8689c4b96e1 (127.8.18.195:38593)
I20260810 11:30:25.060981 10346 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "346a41a691de42858d1a67ab7f25df4c" candidate_uuid: "524610df19f84cfd88b4b7d4c3aaacdc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "600d04a3960143f89a8e1bee5bcd65e7" is_pre_election: true
I20260810 11:30:25.060981 10484 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "346a41a691de42858d1a67ab7f25df4c" candidate_uuid: "524610df19f84cfd88b4b7d4c3aaacdc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2972efc435eb470c88b7c8689c4b96e1" is_pre_election: true
I20260810 11:30:25.061590 10484 raft_consensus.cc:2454] T 346a41a691de42858d1a67ab7f25df4c P 2972efc435eb470c88b7c8689c4b96e1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 524610df19f84cfd88b4b7d4c3aaacdc in term 0.
I20260810 11:30:25.061596 10346 raft_consensus.cc:2454] T 346a41a691de42858d1a67ab7f25df4c P 600d04a3960143f89a8e1bee5bcd65e7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 524610df19f84cfd88b4b7d4c3aaacdc in term 0.
I20260810 11:30:25.062745 10130 leader_election.cc:302] T 346a41a691de42858d1a67ab7f25df4c P 524610df19f84cfd88b4b7d4c3aaacdc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 2972efc435eb470c88b7c8689c4b96e1, 524610df19f84cfd88b4b7d4c3aaacdc, 600d04a3960143f89a8e1bee5bcd65e7; no voters:
I20260810 11:30:25.063438 10653 raft_consensus.cc:2781] T 346a41a691de42858d1a67ab7f25df4c P 524610df19f84cfd88b4b7d4c3aaacdc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 11:30:25.063726 10653 raft_consensus.cc:482] T 346a41a691de42858d1a67ab7f25df4c P 524610df19f84cfd88b4b7d4c3aaacdc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 11:30:25.063995 10653 raft_consensus.cc:3037] T 346a41a691de42858d1a67ab7f25df4c P 524610df19f84cfd88b4b7d4c3aaacdc [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:30:25.069375 10653 raft_consensus.cc:504] T 346a41a691de42858d1a67ab7f25df4c P 524610df19f84cfd88b4b7d4c3aaacdc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "524610df19f84cfd88b4b7d4c3aaacdc" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 35655 } } peers { permanent_uuid: "600d04a3960143f89a8e1bee5bcd65e7" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 45797 } } peers { permanent_uuid: "2972efc435eb470c88b7c8689c4b96e1" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 38593 } }
I20260810 11:30:25.070842 10653 leader_election.cc:288] T 346a41a691de42858d1a67ab7f25df4c P 524610df19f84cfd88b4b7d4c3aaacdc [CANDIDATE]: Term 1 election: Requested vote from peers 600d04a3960143f89a8e1bee5bcd65e7 (127.8.18.194:45797), 2972efc435eb470c88b7c8689c4b96e1 (127.8.18.195:38593)
I20260810 11:30:25.071995 10484 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "346a41a691de42858d1a67ab7f25df4c" candidate_uuid: "524610df19f84cfd88b4b7d4c3aaacdc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2972efc435eb470c88b7c8689c4b96e1"
I20260810 11:30:25.071962 10346 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "346a41a691de42858d1a67ab7f25df4c" candidate_uuid: "524610df19f84cfd88b4b7d4c3aaacdc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "600d04a3960143f89a8e1bee5bcd65e7"
I20260810 11:30:25.072490 10346 raft_consensus.cc:3037] T 346a41a691de42858d1a67ab7f25df4c P 600d04a3960143f89a8e1bee5bcd65e7 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:30:25.072490 10484 raft_consensus.cc:3037] T 346a41a691de42858d1a67ab7f25df4c P 2972efc435eb470c88b7c8689c4b96e1 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:30:25.078625 10346 raft_consensus.cc:2454] T 346a41a691de42858d1a67ab7f25df4c P 600d04a3960143f89a8e1bee5bcd65e7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 524610df19f84cfd88b4b7d4c3aaacdc in term 1.
I20260810 11:30:25.078625 10484 raft_consensus.cc:2454] T 346a41a691de42858d1a67ab7f25df4c P 2972efc435eb470c88b7c8689c4b96e1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 524610df19f84cfd88b4b7d4c3aaacdc in term 1.
I20260810 11:30:25.079727 10131 leader_election.cc:302] T 346a41a691de42858d1a67ab7f25df4c P 524610df19f84cfd88b4b7d4c3aaacdc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2972efc435eb470c88b7c8689c4b96e1, 524610df19f84cfd88b4b7d4c3aaacdc; no voters:
I20260810 11:30:25.080394 10653 raft_consensus.cc:2781] T 346a41a691de42858d1a67ab7f25df4c P 524610df19f84cfd88b4b7d4c3aaacdc [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:30:25.081322 10653 raft_consensus.cc:686] T 346a41a691de42858d1a67ab7f25df4c P 524610df19f84cfd88b4b7d4c3aaacdc [term 1 LEADER]: Becoming Leader. State: Replica: 524610df19f84cfd88b4b7d4c3aaacdc, State: Running, Role: LEADER
I20260810 11:30:25.082216 10653 consensus_queue.cc:237] T 346a41a691de42858d1a67ab7f25df4c P 524610df19f84cfd88b4b7d4c3aaacdc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "524610df19f84cfd88b4b7d4c3aaacdc" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 35655 } } peers { permanent_uuid: "600d04a3960143f89a8e1bee5bcd65e7" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 45797 } } peers { permanent_uuid: "2972efc435eb470c88b7c8689c4b96e1" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 38593 } }
I20260810 11:30:25.088194 10037 catalog_manager.cc:5495] T 346a41a691de42858d1a67ab7f25df4c P 524610df19f84cfd88b4b7d4c3aaacdc reported cstate change: term changed from 0 to 1, leader changed from <none> to 524610df19f84cfd88b4b7d4c3aaacdc (127.8.18.193). New cstate: current_term: 1 leader_uuid: "524610df19f84cfd88b4b7d4c3aaacdc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "524610df19f84cfd88b4b7d4c3aaacdc" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 35655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "600d04a3960143f89a8e1bee5bcd65e7" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 45797 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2972efc435eb470c88b7c8689c4b96e1" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 38593 } health_report { overall_health: UNKNOWN } } }
I20260810 11:30:25.153981 10658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:25.139136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47382 (local address 127.8.18.193:35655)
0810 11:30:25.141286 (+ 2150us) server_negotiation.cc:206] Beginning negotiation
0810 11:30:25.141293 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:30:25.141345 (+ 52us) server_negotiation.cc:407] Connection header received
0810 11:30:25.141504 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:30:25.141509 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:30:25.141623 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:30:25.141781 (+ 158us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 11:30:25.142755 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:25.144512 (+ 1757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:25.148510 (+ 3998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:25.149918 (+ 1408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:25.151578 (+ 1660us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:25.151871 (+ 293us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 11:30:25.152173 (+ 302us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 11:30:25.152316 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0810 11:30:25.152909 (+ 593us) server_negotiation.cc:299] Negotiation successful
0810 11:30:25.153102 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1840,"thread_start_us":1400,"threads_started":1}
I20260810 11:30:25.173310 10484 raft_consensus.cc:1261] T 346a41a691de42858d1a67ab7f25df4c P 2972efc435eb470c88b7c8689c4b96e1 [term 1 FOLLOWER]: Refusing update from remote peer 524610df19f84cfd88b4b7d4c3aaacdc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 11:30:25.173924 10346 raft_consensus.cc:1261] T 346a41a691de42858d1a67ab7f25df4c P 600d04a3960143f89a8e1bee5bcd65e7 [term 1 FOLLOWER]: Refusing update from remote peer 524610df19f84cfd88b4b7d4c3aaacdc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 11:30:25.174578 10655 consensus_queue.cc:1035] T 346a41a691de42858d1a67ab7f25df4c P 524610df19f84cfd88b4b7d4c3aaacdc [LEADER]: Connected to new peer: Peer: permanent_uuid: "2972efc435eb470c88b7c8689c4b96e1" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 38593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260810 11:30:25.175328 10653 consensus_queue.cc:1035] T 346a41a691de42858d1a67ab7f25df4c P 524610df19f84cfd88b4b7d4c3aaacdc [LEADER]: Connected to new peer: Peer: permanent_uuid: "600d04a3960143f89a8e1bee5bcd65e7" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 45797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 11:30:25.277756 10037 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41838:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 11:30:25.278316 10037 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41838:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 11:30:25.285386 10037 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ed0668521a7b4c0e82440311f75dd278: Sending DeleteTablet for 3 replicas of tablet 346a41a691de42858d1a67ab7f25df4c
I20260810 11:30:25.288085 10464 tablet_service.cc:1514] Processing DeleteTablet for tablet 346a41a691de42858d1a67ab7f25df4c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:30:25 UTC) from {username='kudu', principal='kudu/127.8.18.254@KRBTEST.COM'} at 127.0.0.1:40806
I20260810 11:30:25.288223 10178 tablet_service.cc:1514] Processing DeleteTablet for tablet 346a41a691de42858d1a67ab7f25df4c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:30:25 UTC) from {username='kudu', principal='kudu/127.8.18.254@KRBTEST.COM'} at 127.0.0.1:47350
I20260810 11:30:25.288843 10636 tablet_replica.cc:330] T 346a41a691de42858d1a67ab7f25df4c P 2972efc435eb470c88b7c8689c4b96e1: stopping tablet replica
I20260810 11:30:25.289168 10634 tablet_replica.cc:330] T 346a41a691de42858d1a67ab7f25df4c P 524610df19f84cfd88b4b7d4c3aaacdc: stopping tablet replica
I20260810 11:30:25.289865 10634 raft_consensus.cc:2229] T 346a41a691de42858d1a67ab7f25df4c P 524610df19f84cfd88b4b7d4c3aaacdc [term 1 LEADER]: Raft consensus shutting down.
I20260810 11:30:25.290170 10636 raft_consensus.cc:2229] T 346a41a691de42858d1a67ab7f25df4c P 2972efc435eb470c88b7c8689c4b96e1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 11:30:25.290531 10636 raft_consensus.cc:2258] T 346a41a691de42858d1a67ab7f25df4c P 2972efc435eb470c88b7c8689c4b96e1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:30:25.290782 10634 raft_consensus.cc:2258] T 346a41a691de42858d1a67ab7f25df4c P 524610df19f84cfd88b4b7d4c3aaacdc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:30:25.292851 10634 ts_tablet_manager.cc:1905] T 346a41a691de42858d1a67ab7f25df4c P 524610df19f84cfd88b4b7d4c3aaacdc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:30:25.292851 10636 ts_tablet_manager.cc:1905] T 346a41a691de42858d1a67ab7f25df4c P 2972efc435eb470c88b7c8689c4b96e1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:30:25.292199 10037 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41838:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 11:30:25.294255 10037 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 11:30:25.288089 10320 tablet_service.cc:1514] Processing DeleteTablet for tablet 346a41a691de42858d1a67ab7f25df4c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:30:25 UTC) from {username='kudu', principal='kudu/127.8.18.254@KRBTEST.COM'} at 127.0.0.1:47416
I20260810 11:30:25.299675 10635 tablet_replica.cc:330] T 346a41a691de42858d1a67ab7f25df4c P 600d04a3960143f89a8e1bee5bcd65e7: stopping tablet replica
I20260810 11:30:25.300186 10635 raft_consensus.cc:2229] T 346a41a691de42858d1a67ab7f25df4c P 600d04a3960143f89a8e1bee5bcd65e7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 11:30:25.300498 10635 raft_consensus.cc:2258] T 346a41a691de42858d1a67ab7f25df4c P 600d04a3960143f89a8e1bee5bcd65e7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:30:25.303121 10635 ts_tablet_manager.cc:1905] T 346a41a691de42858d1a67ab7f25df4c P 600d04a3960143f89a8e1bee5bcd65e7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:30:25.311184 10636 ts_tablet_manager.cc:1918] T 346a41a691de42858d1a67ab7f25df4c P 2972efc435eb470c88b7c8689c4b96e1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 11:30:25.311699 10636 log.cc:1196] T 346a41a691de42858d1a67ab7f25df4c P 2972efc435eb470c88b7c8689c4b96e1: Deleting WAL directory at /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786361401922362-8267-0/minicluster-data/ts-2/wal/wals/346a41a691de42858d1a67ab7f25df4c
I20260810 11:30:25.312865 10636 ts_tablet_manager.cc:1939] T 346a41a691de42858d1a67ab7f25df4c P 2972efc435eb470c88b7c8689c4b96e1: Deleting consensus metadata
I20260810 11:30:25.315711 10024 catalog_manager.cc:4841] TS 2972efc435eb470c88b7c8689c4b96e1 (127.8.18.195:38593): tablet 346a41a691de42858d1a67ab7f25df4c (table test-table [id=388e1dc1095b4358b5ca1f1b0a98b379]) successfully deleted
I20260810 11:30:25.316665 10320 tablet_service.cc:1467] Processing CreateTablet for tablet 24931d0c35c04499b446b820db011dc0 (DEFAULT_TABLE table=test-table [id=1b1a80c3193842bdb092747742f6522e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:30:25.317130 10178 tablet_service.cc:1467] Processing CreateTablet for tablet 24931d0c35c04499b446b820db011dc0 (DEFAULT_TABLE table=test-table [id=1b1a80c3193842bdb092747742f6522e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:30:25.317868 10320 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 24931d0c35c04499b446b820db011dc0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:30:25.317955 10178 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 24931d0c35c04499b446b820db011dc0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:30:25.317930 10464 tablet_service.cc:1467] Processing CreateTablet for tablet 24931d0c35c04499b446b820db011dc0 (DEFAULT_TABLE table=test-table [id=1b1a80c3193842bdb092747742f6522e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:30:25.318910 10464 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 24931d0c35c04499b446b820db011dc0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:30:25.319547 10634 ts_tablet_manager.cc:1918] T 346a41a691de42858d1a67ab7f25df4c P 524610df19f84cfd88b4b7d4c3aaacdc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 11:30:25.320009 10634 log.cc:1196] T 346a41a691de42858d1a67ab7f25df4c P 524610df19f84cfd88b4b7d4c3aaacdc: Deleting WAL directory at /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786361401922362-8267-0/minicluster-data/ts-0/wal/wals/346a41a691de42858d1a67ab7f25df4c
I20260810 11:30:25.321873 10634 ts_tablet_manager.cc:1939] T 346a41a691de42858d1a67ab7f25df4c P 524610df19f84cfd88b4b7d4c3aaacdc: Deleting consensus metadata
I20260810 11:30:25.324995 10022 catalog_manager.cc:4841] TS 524610df19f84cfd88b4b7d4c3aaacdc (127.8.18.193:35655): tablet 346a41a691de42858d1a67ab7f25df4c (table test-table [id=388e1dc1095b4358b5ca1f1b0a98b379]) successfully deleted
I20260810 11:30:25.334960 10647 tablet_bootstrap.cc:492] T 24931d0c35c04499b446b820db011dc0 P 524610df19f84cfd88b4b7d4c3aaacdc: Bootstrap starting.
I20260810 11:30:25.337096 10645 tablet_bootstrap.cc:492] T 24931d0c35c04499b446b820db011dc0 P 600d04a3960143f89a8e1bee5bcd65e7: Bootstrap starting.
I20260810 11:30:25.337960 10635 ts_tablet_manager.cc:1918] T 346a41a691de42858d1a67ab7f25df4c P 600d04a3960143f89a8e1bee5bcd65e7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 11:30:25.338392 10635 log.cc:1196] T 346a41a691de42858d1a67ab7f25df4c P 600d04a3960143f89a8e1bee5bcd65e7: Deleting WAL directory at /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786361401922362-8267-0/minicluster-data/ts-1/wal/wals/346a41a691de42858d1a67ab7f25df4c
I20260810 11:30:25.339972 10647 tablet_bootstrap.cc:654] T 24931d0c35c04499b446b820db011dc0 P 524610df19f84cfd88b4b7d4c3aaacdc: Neither blocks nor log segments found. Creating new log.
I20260810 11:30:25.340279 10635 ts_tablet_manager.cc:1939] T 346a41a691de42858d1a67ab7f25df4c P 600d04a3960143f89a8e1bee5bcd65e7: Deleting consensus metadata
I20260810 11:30:25.342267 10645 tablet_bootstrap.cc:654] T 24931d0c35c04499b446b820db011dc0 P 600d04a3960143f89a8e1bee5bcd65e7: Neither blocks nor log segments found. Creating new log.
I20260810 11:30:25.342839 10023 catalog_manager.cc:4841] TS 600d04a3960143f89a8e1bee5bcd65e7 (127.8.18.194:45797): tablet 346a41a691de42858d1a67ab7f25df4c (table test-table [id=388e1dc1095b4358b5ca1f1b0a98b379]) successfully deleted
I20260810 11:30:25.346583 10648 tablet_bootstrap.cc:492] T 24931d0c35c04499b446b820db011dc0 P 2972efc435eb470c88b7c8689c4b96e1: Bootstrap starting.
I20260810 11:30:25.348924 10645 tablet_bootstrap.cc:492] T 24931d0c35c04499b446b820db011dc0 P 600d04a3960143f89a8e1bee5bcd65e7: No bootstrap required, opened a new log
I20260810 11:30:25.349463 10645 ts_tablet_manager.cc:1397] T 24931d0c35c04499b446b820db011dc0 P 600d04a3960143f89a8e1bee5bcd65e7: Time spent bootstrapping tablet: real 0.012s user 0.007s sys 0.002s
I20260810 11:30:25.351517 10647 tablet_bootstrap.cc:492] T 24931d0c35c04499b446b820db011dc0 P 524610df19f84cfd88b4b7d4c3aaacdc: No bootstrap required, opened a new log
I20260810 11:30:25.351794 10648 tablet_bootstrap.cc:654] T 24931d0c35c04499b446b820db011dc0 P 2972efc435eb470c88b7c8689c4b96e1: Neither blocks nor log segments found. Creating new log.
I20260810 11:30:25.351990 10647 ts_tablet_manager.cc:1397] T 24931d0c35c04499b446b820db011dc0 P 524610df19f84cfd88b4b7d4c3aaacdc: Time spent bootstrapping tablet: real 0.017s user 0.006s sys 0.009s
I20260810 11:30:25.351871 10645 raft_consensus.cc:348] T 24931d0c35c04499b446b820db011dc0 P 600d04a3960143f89a8e1bee5bcd65e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "524610df19f84cfd88b4b7d4c3aaacdc" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 35655 } } peers { permanent_uuid: "600d04a3960143f89a8e1bee5bcd65e7" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 45797 } } peers { permanent_uuid: "2972efc435eb470c88b7c8689c4b96e1" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 38593 } }
I20260810 11:30:25.352957 10645 raft_consensus.cc:374] T 24931d0c35c04499b446b820db011dc0 P 600d04a3960143f89a8e1bee5bcd65e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:30:25.353271 10645 raft_consensus.cc:729] T 24931d0c35c04499b446b820db011dc0 P 600d04a3960143f89a8e1bee5bcd65e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 600d04a3960143f89a8e1bee5bcd65e7, State: Initialized, Role: FOLLOWER
I20260810 11:30:25.354151 10647 raft_consensus.cc:348] T 24931d0c35c04499b446b820db011dc0 P 524610df19f84cfd88b4b7d4c3aaacdc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "524610df19f84cfd88b4b7d4c3aaacdc" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 35655 } } peers { permanent_uuid: "600d04a3960143f89a8e1bee5bcd65e7" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 45797 } } peers { permanent_uuid: "2972efc435eb470c88b7c8689c4b96e1" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 38593 } }
I20260810 11:30:25.354763 10647 raft_consensus.cc:374] T 24931d0c35c04499b446b820db011dc0 P 524610df19f84cfd88b4b7d4c3aaacdc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:30:25.354584 10645 consensus_queue.cc:260] T 24931d0c35c04499b446b820db011dc0 P 600d04a3960143f89a8e1bee5bcd65e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "524610df19f84cfd88b4b7d4c3aaacdc" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 35655 } } peers { permanent_uuid: "600d04a3960143f89a8e1bee5bcd65e7" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 45797 } } peers { permanent_uuid: "2972efc435eb470c88b7c8689c4b96e1" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 38593 } }
I20260810 11:30:25.354951 10647 raft_consensus.cc:729] T 24931d0c35c04499b446b820db011dc0 P 524610df19f84cfd88b4b7d4c3aaacdc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 524610df19f84cfd88b4b7d4c3aaacdc, State: Initialized, Role: FOLLOWER
I20260810 11:30:25.355515 10647 consensus_queue.cc:260] T 24931d0c35c04499b446b820db011dc0 P 524610df19f84cfd88b4b7d4c3aaacdc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "524610df19f84cfd88b4b7d4c3aaacdc" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 35655 } } peers { permanent_uuid: "600d04a3960143f89a8e1bee5bcd65e7" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 45797 } } peers { permanent_uuid: "2972efc435eb470c88b7c8689c4b96e1" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 38593 } }
I20260810 11:30:25.357138 10647 ts_tablet_manager.cc:1428] T 24931d0c35c04499b446b820db011dc0 P 524610df19f84cfd88b4b7d4c3aaacdc: Time spent starting tablet: real 0.005s user 0.000s sys 0.004s
I20260810 11:30:25.358820 10645 ts_tablet_manager.cc:1428] T 24931d0c35c04499b446b820db011dc0 P 600d04a3960143f89a8e1bee5bcd65e7: Time spent starting tablet: real 0.009s user 0.003s sys 0.002s
I20260810 11:30:25.363250 10648 tablet_bootstrap.cc:492] T 24931d0c35c04499b446b820db011dc0 P 2972efc435eb470c88b7c8689c4b96e1: No bootstrap required, opened a new log
I20260810 11:30:25.363686 10648 ts_tablet_manager.cc:1397] T 24931d0c35c04499b446b820db011dc0 P 2972efc435eb470c88b7c8689c4b96e1: Time spent bootstrapping tablet: real 0.017s user 0.005s sys 0.009s
I20260810 11:30:25.365638 10648 raft_consensus.cc:348] T 24931d0c35c04499b446b820db011dc0 P 2972efc435eb470c88b7c8689c4b96e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "524610df19f84cfd88b4b7d4c3aaacdc" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 35655 } } peers { permanent_uuid: "600d04a3960143f89a8e1bee5bcd65e7" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 45797 } } peers { permanent_uuid: "2972efc435eb470c88b7c8689c4b96e1" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 38593 } }
I20260810 11:30:25.366271 10648 raft_consensus.cc:374] T 24931d0c35c04499b446b820db011dc0 P 2972efc435eb470c88b7c8689c4b96e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:30:25.366482 10648 raft_consensus.cc:729] T 24931d0c35c04499b446b820db011dc0 P 2972efc435eb470c88b7c8689c4b96e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2972efc435eb470c88b7c8689c4b96e1, State: Initialized, Role: FOLLOWER
I20260810 11:30:25.367376 10648 consensus_queue.cc:260] T 24931d0c35c04499b446b820db011dc0 P 2972efc435eb470c88b7c8689c4b96e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "524610df19f84cfd88b4b7d4c3aaacdc" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 35655 } } peers { permanent_uuid: "600d04a3960143f89a8e1bee5bcd65e7" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 45797 } } peers { permanent_uuid: "2972efc435eb470c88b7c8689c4b96e1" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 38593 } }
I20260810 11:30:25.368635 10648 ts_tablet_manager.cc:1428] T 24931d0c35c04499b446b820db011dc0 P 2972efc435eb470c88b7c8689c4b96e1: Time spent starting tablet: real 0.005s user 0.003s sys 0.002s
I20260810 11:30:25.498953 10655 raft_consensus.cc:482] T 24931d0c35c04499b446b820db011dc0 P 524610df19f84cfd88b4b7d4c3aaacdc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 11:30:25.499301 10655 raft_consensus.cc:504] T 24931d0c35c04499b446b820db011dc0 P 524610df19f84cfd88b4b7d4c3aaacdc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "524610df19f84cfd88b4b7d4c3aaacdc" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 35655 } } peers { permanent_uuid: "600d04a3960143f89a8e1bee5bcd65e7" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 45797 } } peers { permanent_uuid: "2972efc435eb470c88b7c8689c4b96e1" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 38593 } }
I20260810 11:30:25.500535 10655 leader_election.cc:288] T 24931d0c35c04499b446b820db011dc0 P 524610df19f84cfd88b4b7d4c3aaacdc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 600d04a3960143f89a8e1bee5bcd65e7 (127.8.18.194:45797), 2972efc435eb470c88b7c8689c4b96e1 (127.8.18.195:38593)
I20260810 11:30:25.501931 10346 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24931d0c35c04499b446b820db011dc0" candidate_uuid: "524610df19f84cfd88b4b7d4c3aaacdc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "600d04a3960143f89a8e1bee5bcd65e7" is_pre_election: true
I20260810 11:30:25.502055 10484 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24931d0c35c04499b446b820db011dc0" candidate_uuid: "524610df19f84cfd88b4b7d4c3aaacdc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2972efc435eb470c88b7c8689c4b96e1" is_pre_election: true
I20260810 11:30:25.502493 10346 raft_consensus.cc:2454] T 24931d0c35c04499b446b820db011dc0 P 600d04a3960143f89a8e1bee5bcd65e7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 524610df19f84cfd88b4b7d4c3aaacdc in term 0.
I20260810 11:30:25.502568 10484 raft_consensus.cc:2454] T 24931d0c35c04499b446b820db011dc0 P 2972efc435eb470c88b7c8689c4b96e1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 524610df19f84cfd88b4b7d4c3aaacdc in term 0.
I20260810 11:30:25.503422 10131 leader_election.cc:302] T 24931d0c35c04499b446b820db011dc0 P 524610df19f84cfd88b4b7d4c3aaacdc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2972efc435eb470c88b7c8689c4b96e1, 524610df19f84cfd88b4b7d4c3aaacdc; no voters:
I20260810 11:30:25.504057 10655 raft_consensus.cc:2781] T 24931d0c35c04499b446b820db011dc0 P 524610df19f84cfd88b4b7d4c3aaacdc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 11:30:25.504263 10655 raft_consensus.cc:482] T 24931d0c35c04499b446b820db011dc0 P 524610df19f84cfd88b4b7d4c3aaacdc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 11:30:25.504469 10655 raft_consensus.cc:3037] T 24931d0c35c04499b446b820db011dc0 P 524610df19f84cfd88b4b7d4c3aaacdc [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:30:25.508924 10655 raft_consensus.cc:504] T 24931d0c35c04499b446b820db011dc0 P 524610df19f84cfd88b4b7d4c3aaacdc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "524610df19f84cfd88b4b7d4c3aaacdc" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 35655 } } peers { permanent_uuid: "600d04a3960143f89a8e1bee5bcd65e7" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 45797 } } peers { permanent_uuid: "2972efc435eb470c88b7c8689c4b96e1" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 38593 } }
I20260810 11:30:25.510324 10655 leader_election.cc:288] T 24931d0c35c04499b446b820db011dc0 P 524610df19f84cfd88b4b7d4c3aaacdc [CANDIDATE]: Term 1 election: Requested vote from peers 600d04a3960143f89a8e1bee5bcd65e7 (127.8.18.194:45797), 2972efc435eb470c88b7c8689c4b96e1 (127.8.18.195:38593)
I20260810 11:30:25.511399 10484 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24931d0c35c04499b446b820db011dc0" candidate_uuid: "524610df19f84cfd88b4b7d4c3aaacdc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2972efc435eb470c88b7c8689c4b96e1"
I20260810 11:30:25.511363 10346 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24931d0c35c04499b446b820db011dc0" candidate_uuid: "524610df19f84cfd88b4b7d4c3aaacdc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "600d04a3960143f89a8e1bee5bcd65e7"
I20260810 11:30:25.511880 10346 raft_consensus.cc:3037] T 24931d0c35c04499b446b820db011dc0 P 600d04a3960143f89a8e1bee5bcd65e7 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:30:25.511960 10484 raft_consensus.cc:3037] T 24931d0c35c04499b446b820db011dc0 P 2972efc435eb470c88b7c8689c4b96e1 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:30:25.518822 10346 raft_consensus.cc:2454] T 24931d0c35c04499b446b820db011dc0 P 600d04a3960143f89a8e1bee5bcd65e7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 524610df19f84cfd88b4b7d4c3aaacdc in term 1.
I20260810 11:30:25.518811 10484 raft_consensus.cc:2454] T 24931d0c35c04499b446b820db011dc0 P 2972efc435eb470c88b7c8689c4b96e1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 524610df19f84cfd88b4b7d4c3aaacdc in term 1.
I20260810 11:30:25.519865 10130 leader_election.cc:302] T 24931d0c35c04499b446b820db011dc0 P 524610df19f84cfd88b4b7d4c3aaacdc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 524610df19f84cfd88b4b7d4c3aaacdc, 600d04a3960143f89a8e1bee5bcd65e7; no voters:
I20260810 11:30:25.520820 10655 raft_consensus.cc:2781] T 24931d0c35c04499b446b820db011dc0 P 524610df19f84cfd88b4b7d4c3aaacdc [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:30:25.521174 10655 raft_consensus.cc:686] T 24931d0c35c04499b446b820db011dc0 P 524610df19f84cfd88b4b7d4c3aaacdc [term 1 LEADER]: Becoming Leader. State: Replica: 524610df19f84cfd88b4b7d4c3aaacdc, State: Running, Role: LEADER
I20260810 11:30:25.521657 10655 consensus_queue.cc:237] T 24931d0c35c04499b446b820db011dc0 P 524610df19f84cfd88b4b7d4c3aaacdc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "524610df19f84cfd88b4b7d4c3aaacdc" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 35655 } } peers { permanent_uuid: "600d04a3960143f89a8e1bee5bcd65e7" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 45797 } } peers { permanent_uuid: "2972efc435eb470c88b7c8689c4b96e1" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 38593 } }
I20260810 11:30:25.527006 10037 catalog_manager.cc:5495] T 24931d0c35c04499b446b820db011dc0 P 524610df19f84cfd88b4b7d4c3aaacdc reported cstate change: term changed from 0 to 1, leader changed from <none> to 524610df19f84cfd88b4b7d4c3aaacdc (127.8.18.193). New cstate: current_term: 1 leader_uuid: "524610df19f84cfd88b4b7d4c3aaacdc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "524610df19f84cfd88b4b7d4c3aaacdc" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 35655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "600d04a3960143f89a8e1bee5bcd65e7" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 45797 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2972efc435eb470c88b7c8689c4b96e1" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 38593 } health_report { overall_health: UNKNOWN } } }
I20260810 11:30:25.591727 10484 raft_consensus.cc:1261] T 24931d0c35c04499b446b820db011dc0 P 2972efc435eb470c88b7c8689c4b96e1 [term 1 FOLLOWER]: Refusing update from remote peer 524610df19f84cfd88b4b7d4c3aaacdc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 11:30:25.591727 10346 raft_consensus.cc:1261] T 24931d0c35c04499b446b820db011dc0 P 600d04a3960143f89a8e1bee5bcd65e7 [term 1 FOLLOWER]: Refusing update from remote peer 524610df19f84cfd88b4b7d4c3aaacdc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 11:30:25.592767 10655 consensus_queue.cc:1035] T 24931d0c35c04499b446b820db011dc0 P 524610df19f84cfd88b4b7d4c3aaacdc [LEADER]: Connected to new peer: Peer: permanent_uuid: "600d04a3960143f89a8e1bee5bcd65e7" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 45797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 11:30:25.593338 10662 consensus_queue.cc:1035] T 24931d0c35c04499b446b820db011dc0 P 524610df19f84cfd88b4b7d4c3aaacdc [LEADER]: Connected to new peer: Peer: permanent_uuid: "2972efc435eb470c88b7c8689c4b96e1" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 38593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260810 11:30:25.637905 10619 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260810 11:30:27.266108 10038 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41838:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 11:30:27.266373 10038 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41838:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 11:30:27.271864 10038 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ed0668521a7b4c0e82440311f75dd278: Sending DeleteTablet for 3 replicas of tablet 24931d0c35c04499b446b820db011dc0
I20260810 11:30:27.273039 10178 tablet_service.cc:1514] Processing DeleteTablet for tablet 24931d0c35c04499b446b820db011dc0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:30:27 UTC) from {username='kudu', principal='kudu/127.8.18.254@KRBTEST.COM'} at 127.0.0.1:47350
I20260810 11:30:27.273126 10320 tablet_service.cc:1514] Processing DeleteTablet for tablet 24931d0c35c04499b446b820db011dc0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:30:27 UTC) from {username='kudu', principal='kudu/127.8.18.254@KRBTEST.COM'} at 127.0.0.1:47416
I20260810 11:30:27.273314 10464 tablet_service.cc:1514] Processing DeleteTablet for tablet 24931d0c35c04499b446b820db011dc0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:30:27 UTC) from {username='kudu', principal='kudu/127.8.18.254@KRBTEST.COM'} at 127.0.0.1:40806
I20260810 11:30:27.274634 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 10112
I20260810 11:30:27.288597 10690 tablet_replica.cc:330] T 24931d0c35c04499b446b820db011dc0 P 524610df19f84cfd88b4b7d4c3aaacdc: stopping tablet replica
I20260810 11:30:27.288933 10690 raft_consensus.cc:2229] T 24931d0c35c04499b446b820db011dc0 P 524610df19f84cfd88b4b7d4c3aaacdc [term 1 LEADER]: Raft consensus shutting down.
I20260810 11:30:27.289561 10690 raft_consensus.cc:2258] T 24931d0c35c04499b446b820db011dc0 P 524610df19f84cfd88b4b7d4c3aaacdc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:30:27.290488 10692 tablet_replica.cc:330] T 24931d0c35c04499b446b820db011dc0 P 2972efc435eb470c88b7c8689c4b96e1: stopping tablet replica
I20260810 11:30:27.290763 10692 raft_consensus.cc:2229] T 24931d0c35c04499b446b820db011dc0 P 2972efc435eb470c88b7c8689c4b96e1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 11:30:27.290908 10690 ts_tablet_manager.cc:1905] T 24931d0c35c04499b446b820db011dc0 P 524610df19f84cfd88b4b7d4c3aaacdc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:30:27.291097 10692 raft_consensus.cc:2258] T 24931d0c35c04499b446b820db011dc0 P 2972efc435eb470c88b7c8689c4b96e1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:30:27.291352 10691 tablet_replica.cc:330] T 24931d0c35c04499b446b820db011dc0 P 600d04a3960143f89a8e1bee5bcd65e7: stopping tablet replica
I20260810 11:30:27.291661 10691 raft_consensus.cc:2229] T 24931d0c35c04499b446b820db011dc0 P 600d04a3960143f89a8e1bee5bcd65e7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 11:30:27.292070 10691 raft_consensus.cc:2258] T 24931d0c35c04499b446b820db011dc0 P 600d04a3960143f89a8e1bee5bcd65e7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:30:27.292723 10692 ts_tablet_manager.cc:1905] T 24931d0c35c04499b446b820db011dc0 P 2972efc435eb470c88b7c8689c4b96e1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:30:27.293329 10691 ts_tablet_manager.cc:1905] T 24931d0c35c04499b446b820db011dc0 P 600d04a3960143f89a8e1bee5bcd65e7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:30:27.300066 10690 ts_tablet_manager.cc:1918] T 24931d0c35c04499b446b820db011dc0 P 524610df19f84cfd88b4b7d4c3aaacdc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260810 11:30:27.300323 10690 log.cc:1196] T 24931d0c35c04499b446b820db011dc0 P 524610df19f84cfd88b4b7d4c3aaacdc: Deleting WAL directory at /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786361401922362-8267-0/minicluster-data/ts-0/wal/wals/24931d0c35c04499b446b820db011dc0
I20260810 11:30:27.300905 10690 ts_tablet_manager.cc:1939] T 24931d0c35c04499b446b820db011dc0 P 524610df19f84cfd88b4b7d4c3aaacdc: Deleting consensus metadata
I20260810 11:30:27.300835 10692 ts_tablet_manager.cc:1918] T 24931d0c35c04499b446b820db011dc0 P 2972efc435eb470c88b7c8689c4b96e1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260810 11:30:27.301106 10692 log.cc:1196] T 24931d0c35c04499b446b820db011dc0 P 2972efc435eb470c88b7c8689c4b96e1: Deleting WAL directory at /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786361401922362-8267-0/minicluster-data/ts-2/wal/wals/24931d0c35c04499b446b820db011dc0
I20260810 11:30:27.301842 10692 ts_tablet_manager.cc:1939] T 24931d0c35c04499b446b820db011dc0 P 2972efc435eb470c88b7c8689c4b96e1: Deleting consensus metadata
I20260810 11:30:27.302349 10691 ts_tablet_manager.cc:1918] T 24931d0c35c04499b446b820db011dc0 P 600d04a3960143f89a8e1bee5bcd65e7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260810 11:30:27.302423 10022 catalog_manager.cc:4841] TS 524610df19f84cfd88b4b7d4c3aaacdc (127.8.18.193:35655): tablet 24931d0c35c04499b446b820db011dc0 (table test-table [id=1b1a80c3193842bdb092747742f6522e]) successfully deleted
I20260810 11:30:27.302665 10691 log.cc:1196] T 24931d0c35c04499b446b820db011dc0 P 600d04a3960143f89a8e1bee5bcd65e7: Deleting WAL directory at /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786361401922362-8267-0/minicluster-data/ts-1/wal/wals/24931d0c35c04499b446b820db011dc0
Aug 10 11:30:27 dist-test-slave-srgv krb5kdc[9992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361424, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.8.18.193@KRBTEST.COM
I20260810 11:30:27.303238 10024 catalog_manager.cc:4841] TS 2972efc435eb470c88b7c8689c4b96e1 (127.8.18.195:38593): tablet 24931d0c35c04499b446b820db011dc0 (table test-table [id=1b1a80c3193842bdb092747742f6522e]) successfully deleted
I20260810 11:30:27.303326 10691 ts_tablet_manager.cc:1939] T 24931d0c35c04499b446b820db011dc0 P 600d04a3960143f89a8e1bee5bcd65e7: Deleting consensus metadata
I20260810 11:30:27.304425 10023 catalog_manager.cc:4841] TS 600d04a3960143f89a8e1bee5bcd65e7 (127.8.18.194:45797): tablet 24931d0c35c04499b446b820db011dc0 (table test-table [id=1b1a80c3193842bdb092747742f6522e]) successfully deleted
I20260810 11:30:27.312346 10694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:27.278536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45568 (local address 127.8.18.193:35655)
0810 11:30:27.282215 (+ 3679us) server_negotiation.cc:206] Beginning negotiation
0810 11:30:27.282226 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 11:30:27.288705 (+ 6479us) server_negotiation.cc:407] Connection header received
0810 11:30:27.289580 (+ 875us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:30:27.289589 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:30:27.289773 (+ 184us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:30:27.289878 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:30:27.292192 (+ 2314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:27.293710 (+ 1518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:27.296022 (+ 2312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:27.296928 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:27.297051 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:27.304373 (+ 7322us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:30:27.304392 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:30:27.304409 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:30:27.304449 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:30:27.308284 (+ 3835us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:27.309267 (+ 983us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:27.309280 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:27.309284 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:27.309594 (+ 310us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:27.310113 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:27.310119 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:27.310122 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:27.310549 (+ 427us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:30:27.310995 (+ 446us) server_negotiation.cc:1035] Waiting for connection context
0810 11:30:27.311472 (+ 477us) server_negotiation.cc:299] Negotiation successful
0810 11:30:27.311804 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3410,"thread_start_us":3135,"threads_started":1}
W20260810 11:30:27.313234 10244 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45568
W20260810 11:30:27.314239 8267 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 10112: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 11:30:27.314499 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 10112
I20260810 11:30:27.339923 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 10256
Aug 10 11:30:27 dist-test-slave-srgv krb5kdc[9992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361424, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.8.18.194@KRBTEST.COM
I20260810 11:30:27.359566 10695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:27.340867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59352 (local address 127.8.18.194:45797)
0810 11:30:27.341723 (+ 856us) server_negotiation.cc:206] Beginning negotiation
0810 11:30:27.341729 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:30:27.341749 (+ 20us) server_negotiation.cc:407] Connection header received
0810 11:30:27.341806 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:30:27.341809 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:30:27.341873 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:30:27.341989 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:30:27.343451 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:27.344514 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:27.346348 (+ 1834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:27.346845 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:27.346963 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:27.352398 (+ 5435us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:30:27.352418 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:30:27.352437 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:30:27.352496 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:30:27.356362 (+ 3866us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:27.357273 (+ 911us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:27.357281 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:27.357284 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:27.357527 (+ 243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:27.357959 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:27.357962 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:27.357965 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:27.358364 (+ 399us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:30:27.358659 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0810 11:30:27.359000 (+ 341us) server_negotiation.cc:299] Negotiation successful
0810 11:30:27.359261 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":729,"thread_start_us":482,"threads_started":1}
W20260810 11:30:27.360420 10387 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59352
W20260810 11:30:27.361204 8267 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 10256: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 11:30:27.361487 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 10256
I20260810 11:30:27.390002 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 10398
Aug 10 11:30:27 dist-test-slave-srgv krb5kdc[9992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361424, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.8.18.195@KRBTEST.COM
I20260810 11:30:27.407740 10696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:27.390938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38632 (local address 127.8.18.195:38593)
0810 11:30:27.391652 (+ 714us) server_negotiation.cc:206] Beginning negotiation
0810 11:30:27.391657 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:30:27.391676 (+ 19us) server_negotiation.cc:407] Connection header received
0810 11:30:27.391735 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:30:27.391739 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:30:27.391810 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:30:27.391914 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:30:27.393340 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:27.394244 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:27.395559 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:27.396029 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:27.396146 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:27.401034 (+ 4888us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:30:27.401053 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:30:27.401116 (+ 63us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:30:27.401162 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:30:27.404351 (+ 3189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:27.405269 (+ 918us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:27.405278 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:27.405281 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:27.405528 (+ 247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:27.406133 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:27.406137 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:27.406139 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:27.406552 (+ 413us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:30:27.406854 (+ 302us) server_negotiation.cc:1035] Waiting for connection context
0810 11:30:27.407253 (+ 399us) server_negotiation.cc:299] Negotiation successful
0810 11:30:27.407458 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":606,"thread_start_us":434,"threads_started":1}
W20260810 11:30:27.408461 10530 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38632
W20260810 11:30:27.409137 8267 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 10398: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 11:30:27.409291 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 10398
I20260810 11:30:27.429581 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 10005
I20260810 11:30:27.431031 10079 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:30:27.555279 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 10005
2026-08-10T11:30:27Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (10162 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786361401922362-8267-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 11:30:27 dist-test-slave-srgv krb5kdc[10700](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 11:30:27 dist-test-slave-srgv krb5kdc[10700](info): set up 2 sockets
Aug 10 11:30:27 dist-test-slave-srgv krb5kdc[10700](info): commencing operation
krb5kdc: starting...
W20260810 11:30:28.689504 8267 mini_kdc.cc:121] Time spent starting KDC: real 1.085s user 0.000s sys 0.024s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786361401922362-8267-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786361401922362-8267-0/krb5kdc/test-user.keytab.
Aug 10 11:30:28 dist-test-slave-srgv krb5kdc[10700](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361428, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T11:30:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T11:30:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.18.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.18.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.18.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.254.keytab.
Entry for principal kudu/127.8.18.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.254.keytab.
I20260810 11:30:28.925657 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786361401922362-8267-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786361401922362-8267-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786361401922362-8267-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.18.254:42601
--webserver_interface=127.8.18.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.254.keytab
--principal=kudu/127.8.18.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.18.212:43951
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.18.254:42601
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786361401922362-8267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786361401922362-8267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786361401922362-8267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:30:29.280064 10713 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:30:29.280464 10713 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:30:29.280651 10713 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:30:29.288713 10713 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 11:30:29.288895 10713 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:30:29.289009 10713 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:30:29.289141 10713 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 11:30:29.289223 10713 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 11:30:29.299959 10713 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.18.212:43951
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786361401922362-8267-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786361401922362-8267-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.18.254:42601
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.254.keytab
--principal=kudu/127.8.18.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.18.254:42601
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.18.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786361401922362-8267-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:30:29.301491 10713 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:30:29.303051 10713 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:30:29.317077 10721 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:30:29.317162 10719 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:30:29.317682 10718 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:30:29.318102 10713 server_base.cc:1034] running on GCE node
I20260810 11:30:29.319026 10713 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:30:29.320686 10713 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:30:29.321931 10713 hybrid_clock.cc:648] HybridClock initialized: now 1786361429321851 us; error 62 us; skew 500 ppm
I20260810 11:30:29.322376 10713 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:30:29 dist-test-slave-srgv krb5kdc[10700](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361429, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:30:29.326186 10713 init.cc:377] Logged in from keytab as kudu/127.8.18.254@KRBTEST.COM (short username kudu)
I20260810 11:30:29.328186 10713 webserver.cc:460] Webserver started at http://127.8.18.254:46423/ using document root <none> and password file <none>
I20260810 11:30:29.329030 10713 fs_manager.cc:359] Metadata directory not provided
I20260810 11:30:29.329202 10713 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:30:29.329648 10713 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:30:29.333056 10713 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786361401922362-8267-0/minicluster-data/master-0/data/instance:
uuid: "f259f98b3b444a91959ccf1d02dc05cd"
format_stamp: "Formatted at 2026-08-10 11:30:29 on dist-test-slave-srgv"
server_key: "3695c768f9112dc42985c38e34420d85"
server_key_iv: "fad055d6a865b215dd26d534094d8d17"
server_key_version: "encryptionkey@0"
I20260810 11:30:29.334045 10713 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786361401922362-8267-0/minicluster-data/master-0/wal/instance:
uuid: "f259f98b3b444a91959ccf1d02dc05cd"
format_stamp: "Formatted at 2026-08-10 11:30:29 on dist-test-slave-srgv"
server_key: "3695c768f9112dc42985c38e34420d85"
server_key_iv: "fad055d6a865b215dd26d534094d8d17"
server_key_version: "encryptionkey@0"
I20260810 11:30:29.340113 10713 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260810 11:30:29.344282 10728 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:30:29.345995 10713 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 11:30:29.346222 10713 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786361401922362-8267-0/minicluster-data/master-0/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786361401922362-8267-0/minicluster-data/master-0/wal
uuid: "f259f98b3b444a91959ccf1d02dc05cd"
format_stamp: "Formatted at 2026-08-10 11:30:29 on dist-test-slave-srgv"
server_key: "3695c768f9112dc42985c38e34420d85"
server_key_iv: "fad055d6a865b215dd26d534094d8d17"
server_key_version: "encryptionkey@0"
I20260810 11:30:29.346462 10713 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786361401922362-8267-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786361401922362-8267-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786361401922362-8267-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:30:29.364940 10713 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:30:29.370422 10713 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:30:29.370811 10713 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:30:29.392865 10713 rpc_server.cc:307] RPC server started. Bound to: 127.8.18.254:42601
I20260810 11:30:29.392897 10780 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.18.254:42601 every 8 connection(s)
I20260810 11:30:29.394526 10713 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
I20260810 11:30:29.398855 10781 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:30:29.400328 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 10713
I20260810 11:30:29.400624 8267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786361401922362-8267-0/minicluster-data/master-0/wal/instance
I20260810 11:30:29.401731 8267 external_mini_cluster.cc:1428] Setting key 1cbfed42d33b07ee03afe9a41e6827af
I20260810 11:30:29.416754 10781 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f259f98b3b444a91959ccf1d02dc05cd: Bootstrap starting.
Aug 10 11:30:29 dist-test-slave-srgv krb5kdc[10700](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361428, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.18.254@KRBTEST.COM
I20260810 11:30:29.421203 10781 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f259f98b3b444a91959ccf1d02dc05cd: Neither blocks nor log segments found. Creating new log.
I20260810 11:30:29.422578 10781 log.cc:824] T 00000000000000000000000000000000 P f259f98b3b444a91959ccf1d02dc05cd: Log is configured to *not* fsync() on all Append() calls
I20260810 11:30:29.426537 10781 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f259f98b3b444a91959ccf1d02dc05cd: No bootstrap required, opened a new log
I20260810 11:30:29.430855 10784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:29.404360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55312 (local address 127.8.18.254:42601)
0810 11:30:29.405377 (+ 1017us) server_negotiation.cc:206] Beginning negotiation
0810 11:30:29.405391 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 11:30:29.405507 (+ 116us) server_negotiation.cc:407] Connection header received
0810 11:30:29.407333 (+ 1826us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:30:29.407363 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:30:29.407881 (+ 518us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:30:29.408314 (+ 433us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:30:29.410559 (+ 2245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:29.412049 (+ 1490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:29.414279 (+ 2230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:29.415156 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:29.415357 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:29.421029 (+ 5672us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:30:29.421041 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:30:29.421058 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:30:29.421109 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:30:29.425360 (+ 4251us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:29.426302 (+ 942us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:29.426312 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:29.426321 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:29.426577 (+ 256us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:29.427050 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:29.427055 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:29.427059 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:29.427912 (+ 853us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:30:29.428279 (+ 367us) server_negotiation.cc:1035] Waiting for connection context
0810 11:30:29.428656 (+ 377us) server_negotiation.cc:299] Negotiation successful
0810 11:30:29.429205 (+ 549us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":750,"thread_start_us":504,"threads_started":1}
I20260810 11:30:29.436867 10781 raft_consensus.cc:348] T 00000000000000000000000000000000 P f259f98b3b444a91959ccf1d02dc05cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f259f98b3b444a91959ccf1d02dc05cd" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 42601 } }
I20260810 11:30:29.437307 10781 raft_consensus.cc:374] T 00000000000000000000000000000000 P f259f98b3b444a91959ccf1d02dc05cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:30:29.437390 10781 raft_consensus.cc:729] T 00000000000000000000000000000000 P f259f98b3b444a91959ccf1d02dc05cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f259f98b3b444a91959ccf1d02dc05cd, State: Initialized, Role: FOLLOWER
I20260810 11:30:29.438140 10781 consensus_queue.cc:260] T 00000000000000000000000000000000 P f259f98b3b444a91959ccf1d02dc05cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f259f98b3b444a91959ccf1d02dc05cd" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 42601 } }
I20260810 11:30:29.438388 10781 raft_consensus.cc:388] T 00000000000000000000000000000000 P f259f98b3b444a91959ccf1d02dc05cd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 11:30:29.438490 10781 raft_consensus.cc:482] T 00000000000000000000000000000000 P f259f98b3b444a91959ccf1d02dc05cd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 11:30:29.438617 10781 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f259f98b3b444a91959ccf1d02dc05cd [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:30:29.441228 10781 raft_consensus.cc:504] T 00000000000000000000000000000000 P f259f98b3b444a91959ccf1d02dc05cd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f259f98b3b444a91959ccf1d02dc05cd" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 42601 } }
I20260810 11:30:29.441913 10781 leader_election.cc:302] T 00000000000000000000000000000000 P f259f98b3b444a91959ccf1d02dc05cd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f259f98b3b444a91959ccf1d02dc05cd; no voters:
I20260810 11:30:29.442605 10781 leader_election.cc:288] T 00000000000000000000000000000000 P f259f98b3b444a91959ccf1d02dc05cd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 11:30:29.442785 10786 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f259f98b3b444a91959ccf1d02dc05cd [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:30:29.443593 10786 raft_consensus.cc:686] T 00000000000000000000000000000000 P f259f98b3b444a91959ccf1d02dc05cd [term 1 LEADER]: Becoming Leader. State: Replica: f259f98b3b444a91959ccf1d02dc05cd, State: Running, Role: LEADER
I20260810 11:30:29.444243 10786 consensus_queue.cc:237] T 00000000000000000000000000000000 P f259f98b3b444a91959ccf1d02dc05cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f259f98b3b444a91959ccf1d02dc05cd" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 42601 } }
I20260810 11:30:29.445052 10781 sys_catalog.cc:564] T 00000000000000000000000000000000 P f259f98b3b444a91959ccf1d02dc05cd [sys.catalog]: configured and running, proceeding with master startup.
I20260810 11:30:29.447947 10787 sys_catalog.cc:455] T 00000000000000000000000000000000 P f259f98b3b444a91959ccf1d02dc05cd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f259f98b3b444a91959ccf1d02dc05cd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f259f98b3b444a91959ccf1d02dc05cd" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 42601 } } }
I20260810 11:30:29.448515 10787 sys_catalog.cc:458] T 00000000000000000000000000000000 P f259f98b3b444a91959ccf1d02dc05cd [sys.catalog]: This master's current role is: LEADER
I20260810 11:30:29.447930 10788 sys_catalog.cc:455] T 00000000000000000000000000000000 P f259f98b3b444a91959ccf1d02dc05cd [sys.catalog]: SysCatalogTable state changed. Reason: New leader f259f98b3b444a91959ccf1d02dc05cd. Latest consensus state: current_term: 1 leader_uuid: "f259f98b3b444a91959ccf1d02dc05cd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f259f98b3b444a91959ccf1d02dc05cd" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 42601 } } }
I20260810 11:30:29.457815 10788 sys_catalog.cc:458] T 00000000000000000000000000000000 P f259f98b3b444a91959ccf1d02dc05cd [sys.catalog]: This master's current role is: LEADER
I20260810 11:30:29.461282 10793 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 11:30:29.466455 10793 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 11:30:29.476390 10793 catalog_manager.cc:1347] Generated new cluster ID: 1a13bddbae2b42dfa5fac85eda4195fd
I20260810 11:30:29.476586 10793 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 11:30:29.491099 10793 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 11:30:29.492676 10793 catalog_manager.cc:1504] Loading token signing keys...
I20260810 11:30:29.501173 10793 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f259f98b3b444a91959ccf1d02dc05cd: Generated new TSK 0
I20260810 11:30:29.502236 10793 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.18.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.18.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.18.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.193.keytab.
Entry for principal kudu/127.8.18.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.193.keytab.
I20260810 11:30:29.561857 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786361401922362-8267-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786361401922362-8267-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786361401922362-8267-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786361401922362-8267-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.18.193:0
--local_ip_for_outbound_sockets=127.8.18.193
--webserver_interface=127.8.18.193
--webserver_port=0
--tserver_master_addrs=127.8.18.254:42601
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.193.keytab
--principal=kudu/127.8.18.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.18.212:43951
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786361401922362-8267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786361401922362-8267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786361401922362-8267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:30:29.934670 10808 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:30:29.934995 10808 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:30:29.935107 10808 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:30:29.942838 10808 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:30:29.942986 10808 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:30:29.943117 10808 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.18.193
I20260810 11:30:29.957607 10808 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.18.212:43951
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786361401922362-8267-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786361401922362-8267-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.193.keytab
--principal=kudu/127.8.18.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.18.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786361401922362-8267-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.18.193
--webserver_port=0
--tserver_master_addrs=127.8.18.254:42601
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.18.193
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786361401922362-8267-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:30:29.959260 10808 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:30:29.961138 10808 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:30:29.975788 10814 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:30:29.975883 10816 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:30:29.975879 10813 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:30:29.976967 10808 server_base.cc:1034] running on GCE node
I20260810 11:30:29.977766 10808 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:30:29.978873 10808 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:30:29.980135 10808 hybrid_clock.cc:648] HybridClock initialized: now 1786361429980103 us; error 69 us; skew 500 ppm
I20260810 11:30:29.980518 10808 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:30:29 dist-test-slave-srgv krb5kdc[10700](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361429, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:30:29.988628 10808 init.cc:377] Logged in from keytab as kudu/127.8.18.193@KRBTEST.COM (short username kudu)
I20260810 11:30:29.990602 10808 webserver.cc:460] Webserver started at http://127.8.18.193:43407/ using document root <none> and password file <none>
I20260810 11:30:29.991542 10808 fs_manager.cc:359] Metadata directory not provided
I20260810 11:30:29.991720 10808 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:30:29.992084 10808 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:30:29.995730 10808 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786361401922362-8267-0/minicluster-data/ts-0/data/instance:
uuid: "b6212d1c51ae42f49123ecbceb18fe76"
format_stamp: "Formatted at 2026-08-10 11:30:29 on dist-test-slave-srgv"
server_key: "81a9394dcc87af358305c387b1000a28"
server_key_iv: "b25a358bd09217f9614babaee43a4340"
server_key_version: "encryptionkey@0"
I20260810 11:30:29.996709 10808 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786361401922362-8267-0/minicluster-data/ts-0/wal/instance:
uuid: "b6212d1c51ae42f49123ecbceb18fe76"
format_stamp: "Formatted at 2026-08-10 11:30:29 on dist-test-slave-srgv"
server_key: "81a9394dcc87af358305c387b1000a28"
server_key_iv: "b25a358bd09217f9614babaee43a4340"
server_key_version: "encryptionkey@0"
I20260810 11:30:30.003005 10808 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260810 11:30:30.007426 10823 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:30:30.009183 10808 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.000s
I20260810 11:30:30.009541 10808 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786361401922362-8267-0/minicluster-data/ts-0/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786361401922362-8267-0/minicluster-data/ts-0/wal
uuid: "b6212d1c51ae42f49123ecbceb18fe76"
format_stamp: "Formatted at 2026-08-10 11:30:29 on dist-test-slave-srgv"
server_key: "81a9394dcc87af358305c387b1000a28"
server_key_iv: "b25a358bd09217f9614babaee43a4340"
server_key_version: "encryptionkey@0"
I20260810 11:30:30.009814 10808 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786361401922362-8267-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786361401922362-8267-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786361401922362-8267-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:30:30.040076 10808 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:30:30.045469 10808 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:30:30.045933 10808 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:30:30.047165 10808 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:30:30.048768 10808 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:30:30.048885 10808 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:30:30.049048 10808 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:30:30.049125 10808 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:30:30.087419 10808 rpc_server.cc:307] RPC server started. Bound to: 127.8.18.193:34789
I20260810 11:30:30.087484 10939 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.18.193:34789 every 8 connection(s)
I20260810 11:30:30.088804 10808 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786361401922362-8267-0/minicluster-data/ts-0/data/info.pb
I20260810 11:30:30.097345 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 10808
I20260810 11:30:30.097653 8267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786361401922362-8267-0/minicluster-data/ts-0/wal/instance
I20260810 11:30:30.098423 8267 external_mini_cluster.cc:1428] Setting key ab831367e6ad851fa92fe9ad9b2a2002
Aug 10 11:30:30 dist-test-slave-srgv krb5kdc[10700](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361429, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.193@KRBTEST.COM for kudu/127.8.18.254@KRBTEST.COM
I20260810 11:30:30.115003 10943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:30.091718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.193:44973 (local address 127.8.18.254:42601)
0810 11:30:30.092200 (+ 482us) server_negotiation.cc:206] Beginning negotiation
0810 11:30:30.092205 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:30:30.093313 (+ 1108us) server_negotiation.cc:407] Connection header received
0810 11:30:30.095374 (+ 2061us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:30:30.095380 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:30:30.095483 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:30:30.095613 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:30:30.097320 (+ 1707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:30.098289 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:30.100474 (+ 2185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:30.101149 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:30.101313 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:30.106239 (+ 4926us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:30:30.106251 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:30:30.106258 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:30:30.106314 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:30:30.110350 (+ 4036us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:30.111401 (+ 1051us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:30.111411 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:30.111417 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:30.111613 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:30.112498 (+ 885us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:30.112507 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:30.112513 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:30.112965 (+ 452us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:30:30.113145 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0810 11:30:30.114183 (+ 1038us) server_negotiation.cc:299] Negotiation successful
0810 11:30:30.114491 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":352,"thread_start_us":239,"threads_started":1}
I20260810 11:30:30.116156 10944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:30.092339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:42601 (local address 127.8.18.193:44973)
0810 11:30:30.093106 (+ 767us) negotiation.cc:107] Waiting for socket to connect
0810 11:30:30.093146 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0810 11:30:30.095034 (+ 1888us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:30:30.095902 (+ 868us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:30:30.095911 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:30:30.096500 (+ 589us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:30:30.097103 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:30:30.097121 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:30:30.098488 (+ 1367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:30:30.098492 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:30:30.100259 (+ 1767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:30:30.100283 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:30:30.101474 (+ 1191us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:30:30.101482 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:30:30.101712 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:30.102517 (+ 805us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:30:30.102542 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:30:30.105847 (+ 3305us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:30:30.110605 (+ 4758us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:30:30.110614 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:30:30.110631 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:30:30.111224 (+ 593us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:30:30.111831 (+ 607us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:30:30.111835 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:30:30.111838 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:30:30.112335 (+ 497us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:30:30.113122 (+ 787us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:30:30.113129 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:30:30.113721 (+ 592us) client_negotiation.cc:769] Sending connection context
0810 11:30:30.114054 (+ 333us) client_negotiation.cc:240] Negotiation successful
0810 11:30:30.114573 (+ 519us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":534,"thread_start_us":315,"threads_started":1}
I20260810 11:30:30.117985 10940 heartbeater.cc:347] Connected to a master server at 127.8.18.254:42601
I20260810 11:30:30.118355 10940 heartbeater.cc:464] Registering TS with master...
I20260810 11:30:30.119128 10940 heartbeater.cc:511] Master 127.8.18.254:42601 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.8.18.194@KRBTEST.COM; defaulting to no policy
I20260810 11:30:30.121495 10745 ts_manager.cc:194] Registered new tserver with Master: b6212d1c51ae42f49123ecbceb18fe76 (127.8.18.193:34789)
I20260810 11:30:30.123533 10745 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.18.193@KRBTEST.COM'} at 127.8.18.193:44973
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.18.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.18.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.194.keytab.
Entry for principal kudu/127.8.18.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.194.keytab.
I20260810 11:30:30.153692 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786361401922362-8267-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786361401922362-8267-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786361401922362-8267-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786361401922362-8267-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.18.194:0
--local_ip_for_outbound_sockets=127.8.18.194
--webserver_interface=127.8.18.194
--webserver_port=0
--tserver_master_addrs=127.8.18.254:42601
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.194.keytab
--principal=kudu/127.8.18.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.18.212:43951
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786361401922362-8267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786361401922362-8267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786361401922362-8267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:30:30.519440 10947 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:30:30.519829 10947 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:30:30.520009 10947 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:30:30.527876 10947 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:30:30.528072 10947 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:30:30.528276 10947 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.18.194
I20260810 11:30:30.539081 10947 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.18.212:43951
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786361401922362-8267-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786361401922362-8267-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.194.keytab
--principal=kudu/127.8.18.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.18.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786361401922362-8267-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.18.194
--webserver_port=0
--tserver_master_addrs=127.8.18.254:42601
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.18.194
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786361401922362-8267-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:30:30.540632 10947 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:30:30.542150 10947 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:30:30.558182 10953 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:30:30.558189 10954 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:30:30.558138 10956 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:30:30.559190 10947 server_base.cc:1034] running on GCE node
I20260810 11:30:30.559933 10947 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:30:30.561020 10947 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:30:30.562291 10947 hybrid_clock.cc:648] HybridClock initialized: now 1786361430562239 us; error 52 us; skew 500 ppm
I20260810 11:30:30.562683 10947 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:30:30 dist-test-slave-srgv krb5kdc[10700](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361430, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:30:30.566433 10947 init.cc:377] Logged in from keytab as kudu/127.8.18.194@KRBTEST.COM (short username kudu)
I20260810 11:30:30.568176 10947 webserver.cc:460] Webserver started at http://127.8.18.194:36451/ using document root <none> and password file <none>
I20260810 11:30:30.569003 10947 fs_manager.cc:359] Metadata directory not provided
I20260810 11:30:30.569180 10947 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:30:30.569614 10947 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:30:30.572877 10947 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786361401922362-8267-0/minicluster-data/ts-1/data/instance:
uuid: "a95bee3b052547849b77cc1cf51f8710"
format_stamp: "Formatted at 2026-08-10 11:30:30 on dist-test-slave-srgv"
server_key: "b599b4bffc7c4e7455de9e9c93cdf1f8"
server_key_iv: "3b4efcd41cf757c696c0c006ce195115"
server_key_version: "encryptionkey@0"
I20260810 11:30:30.573876 10947 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786361401922362-8267-0/minicluster-data/ts-1/wal/instance:
uuid: "a95bee3b052547849b77cc1cf51f8710"
format_stamp: "Formatted at 2026-08-10 11:30:30 on dist-test-slave-srgv"
server_key: "b599b4bffc7c4e7455de9e9c93cdf1f8"
server_key_iv: "3b4efcd41cf757c696c0c006ce195115"
server_key_version: "encryptionkey@0"
I20260810 11:30:30.579926 10947 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.001s sys 0.004s
I20260810 11:30:30.584154 10963 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:30:30.586027 10947 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260810 11:30:30.586292 10947 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786361401922362-8267-0/minicluster-data/ts-1/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786361401922362-8267-0/minicluster-data/ts-1/wal
uuid: "a95bee3b052547849b77cc1cf51f8710"
format_stamp: "Formatted at 2026-08-10 11:30:30 on dist-test-slave-srgv"
server_key: "b599b4bffc7c4e7455de9e9c93cdf1f8"
server_key_iv: "3b4efcd41cf757c696c0c006ce195115"
server_key_version: "encryptionkey@0"
I20260810 11:30:30.586560 10947 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786361401922362-8267-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786361401922362-8267-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786361401922362-8267-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:30:30.626681 10947 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:30:30.631618 10947 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:30:30.631980 10947 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:30:30.633134 10947 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:30:30.634786 10947 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:30:30.634924 10947 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:30:30.635097 10947 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:30:30.635185 10947 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:30:30.672590 10947 rpc_server.cc:307] RPC server started. Bound to: 127.8.18.194:46415
I20260810 11:30:30.672658 11077 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.18.194:46415 every 8 connection(s)
I20260810 11:30:30.674135 10947 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786361401922362-8267-0/minicluster-data/ts-1/data/info.pb
I20260810 11:30:30.678797 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 10947
I20260810 11:30:30.679009 8267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786361401922362-8267-0/minicluster-data/ts-1/wal/instance
I20260810 11:30:30.679632 8267 external_mini_cluster.cc:1428] Setting key 9fb39e95d656645e7ff4b4b6b9e7dbd2
Aug 10 11:30:30 dist-test-slave-srgv krb5kdc[10700](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361430, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.194@KRBTEST.COM for kudu/127.8.18.254@KRBTEST.COM
I20260810 11:30:30.702123 11080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:30.676955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.194:57497 (local address 127.8.18.254:42601)
0810 11:30:30.677520 (+ 565us) server_negotiation.cc:206] Beginning negotiation
0810 11:30:30.677525 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:30:30.678497 (+ 972us) server_negotiation.cc:407] Connection header received
0810 11:30:30.681000 (+ 2503us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:30:30.681005 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:30:30.681076 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:30:30.681180 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:30:30.683514 (+ 2334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:30.684592 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:30.687612 (+ 3020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:30.688332 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:30.688472 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:30.693361 (+ 4889us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:30:30.693373 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:30:30.693380 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:30:30.693491 (+ 111us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:30:30.697651 (+ 4160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:30.698708 (+ 1057us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:30.698718 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:30.698724 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:30.698934 (+ 210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:30.699487 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:30.699495 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:30.699502 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:30.699971 (+ 469us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:30:30.700160 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0810 11:30:30.701183 (+ 1023us) server_negotiation.cc:299] Negotiation successful
0810 11:30:30.701555 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":393,"thread_start_us":280,"threads_started":1}
I20260810 11:30:30.702978 11081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:30.677474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:42601 (local address 127.8.18.194:57497)
0810 11:30:30.678241 (+ 767us) negotiation.cc:107] Waiting for socket to connect
0810 11:30:30.678286 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0810 11:30:30.680673 (+ 2387us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:30:30.681518 (+ 845us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:30:30.681531 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:30:30.682291 (+ 760us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:30:30.683223 (+ 932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:30:30.683262 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:30:30.684813 (+ 1551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:30:30.684822 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:30:30.687325 (+ 2503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:30:30.687353 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:30:30.688667 (+ 1314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:30:30.688678 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:30:30.689082 (+ 404us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:30.690103 (+ 1021us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:30:30.690132 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:30:30.692940 (+ 2808us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:30:30.697916 (+ 4976us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:30:30.697927 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:30:30.697945 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:30:30.698507 (+ 562us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:30:30.699073 (+ 566us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:30:30.699077 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:30:30.699079 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:30:30.699316 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:30:30.700158 (+ 842us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:30:30.700165 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:30:30.700682 (+ 517us) client_negotiation.cc:769] Sending connection context
0810 11:30:30.700972 (+ 290us) client_negotiation.cc:240] Negotiation successful
0810 11:30:30.701572 (+ 600us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":510,"thread_start_us":304,"threads_started":1}
WARNING: no policy specified for kudu/127.8.18.195@KRBTEST.COM; defaulting to no policy
I20260810 11:30:30.704802 11078 heartbeater.cc:347] Connected to a master server at 127.8.18.254:42601
I20260810 11:30:30.705215 11078 heartbeater.cc:464] Registering TS with master...
I20260810 11:30:30.706151 11078 heartbeater.cc:511] Master 127.8.18.254:42601 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.18.195@KRBTEST.COM" created.
I20260810 11:30:30.708163 10745 ts_manager.cc:194] Registered new tserver with Master: a95bee3b052547849b77cc1cf51f8710 (127.8.18.194:46415)
I20260810 11:30:30.709118 10745 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.18.194@KRBTEST.COM'} at 127.8.18.194:57497
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.18.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.195.keytab.
Entry for principal kudu/127.8.18.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.195.keytab.
I20260810 11:30:30.737697 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786361401922362-8267-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786361401922362-8267-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786361401922362-8267-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786361401922362-8267-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.18.195:0
--local_ip_for_outbound_sockets=127.8.18.195
--webserver_interface=127.8.18.195
--webserver_port=0
--tserver_master_addrs=127.8.18.254:42601
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.195.keytab
--principal=kudu/127.8.18.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.18.212:43951
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786361401922362-8267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786361401922362-8267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786361401922362-8267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:30:31.109688 11084 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:30:31.110028 11084 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:30:31.110204 11084 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:30:31.118273 11084 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:30:31.118420 11084 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:30:31.118626 11084 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.18.195
I20260810 11:30:31.127328 10940 heartbeater.cc:503] Master 127.8.18.254:42601 was elected leader, sending a full tablet report...
I20260810 11:30:31.130216 11084 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.18.212:43951
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786361401922362-8267-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786361401922362-8267-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.195.keytab
--principal=kudu/127.8.18.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.18.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786361401922362-8267-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.18.195
--webserver_port=0
--tserver_master_addrs=127.8.18.254:42601
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.18.195
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786361401922362-8267-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:30:31.131716 11084 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:30:31.133138 11084 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:30:31.151787 11091 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:30:31.152163 11090 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:30:31.153100 11084 server_base.cc:1034] running on GCE node
W20260810 11:30:31.153592 11093 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:30:31.154405 11084 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:30:31.155601 11084 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:30:31.156895 11084 hybrid_clock.cc:648] HybridClock initialized: now 1786361431156815 us; error 79 us; skew 500 ppm
I20260810 11:30:31.157274 11084 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:30:31 dist-test-slave-srgv krb5kdc[10700](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361431, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:30:31.161276 11084 init.cc:377] Logged in from keytab as kudu/127.8.18.195@KRBTEST.COM (short username kudu)
I20260810 11:30:31.163197 11084 webserver.cc:460] Webserver started at http://127.8.18.195:43255/ using document root <none> and password file <none>
I20260810 11:30:31.164140 11084 fs_manager.cc:359] Metadata directory not provided
I20260810 11:30:31.164335 11084 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:30:31.164719 11084 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:30:31.168224 11084 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786361401922362-8267-0/minicluster-data/ts-2/data/instance:
uuid: "b5a9c24f60ae4326b4f0c0c210850815"
format_stamp: "Formatted at 2026-08-10 11:30:31 on dist-test-slave-srgv"
server_key: "2d108a84df7bdd31f3dad24a2c933f98"
server_key_iv: "2313ca0e99b001f83c87018c8e378c5f"
server_key_version: "encryptionkey@0"
I20260810 11:30:31.169188 11084 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786361401922362-8267-0/minicluster-data/ts-2/wal/instance:
uuid: "b5a9c24f60ae4326b4f0c0c210850815"
format_stamp: "Formatted at 2026-08-10 11:30:31 on dist-test-slave-srgv"
server_key: "2d108a84df7bdd31f3dad24a2c933f98"
server_key_iv: "2313ca0e99b001f83c87018c8e378c5f"
server_key_version: "encryptionkey@0"
I20260810 11:30:31.175406 11084 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.000s sys 0.004s
I20260810 11:30:31.180431 11100 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:30:31.182193 11084 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260810 11:30:31.182404 11084 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786361401922362-8267-0/minicluster-data/ts-2/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786361401922362-8267-0/minicluster-data/ts-2/wal
uuid: "b5a9c24f60ae4326b4f0c0c210850815"
format_stamp: "Formatted at 2026-08-10 11:30:31 on dist-test-slave-srgv"
server_key: "2d108a84df7bdd31f3dad24a2c933f98"
server_key_iv: "2313ca0e99b001f83c87018c8e378c5f"
server_key_version: "encryptionkey@0"
I20260810 11:30:31.182654 11084 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786361401922362-8267-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786361401922362-8267-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786361401922362-8267-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:30:31.209718 11084 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:30:31.215024 11084 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:30:31.215415 11084 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:30:31.216601 11084 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:30:31.218655 11084 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:30:31.218752 11084 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:30:31.218935 11084 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:30:31.219004 11084 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:30:31.256243 11084 rpc_server.cc:307] RPC server started. Bound to: 127.8.18.195:45777
I20260810 11:30:31.256362 11214 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.18.195:45777 every 8 connection(s)
I20260810 11:30:31.257900 11084 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786361401922362-8267-0/minicluster-data/ts-2/data/info.pb
I20260810 11:30:31.263731 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 11084
I20260810 11:30:31.263887 8267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786361401922362-8267-0/minicluster-data/ts-2/wal/instance
I20260810 11:30:31.264400 8267 external_mini_cluster.cc:1428] Setting key 073aa0aef551f71bd9f0f86006b915b2
Aug 10 11:30:31 dist-test-slave-srgv krb5kdc[10700](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361431, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.195@KRBTEST.COM for kudu/127.8.18.254@KRBTEST.COM
I20260810 11:30:31.283012 11217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:31.260976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.195:32769 (local address 127.8.18.254:42601)
0810 11:30:31.261556 (+ 580us) server_negotiation.cc:206] Beginning negotiation
0810 11:30:31.261561 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:30:31.262810 (+ 1249us) server_negotiation.cc:407] Connection header received
0810 11:30:31.265200 (+ 2390us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:30:31.265206 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:30:31.265327 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:30:31.265499 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:30:31.267710 (+ 2211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:31.268924 (+ 1214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:31.270657 (+ 1733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:31.271158 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:31.271313 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:31.275808 (+ 4495us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:30:31.275814 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:30:31.275818 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:30:31.275860 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:30:31.278914 (+ 3054us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:31.280138 (+ 1224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:31.280149 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:31.280153 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:31.280343 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:31.281013 (+ 670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:31.281019 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:31.281021 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:31.281452 (+ 431us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:30:31.281632 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0810 11:30:31.282500 (+ 868us) server_negotiation.cc:299] Negotiation successful
0810 11:30:31.282703 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":382,"thread_start_us":242,"threads_started":1}
I20260810 11:30:31.284099 11218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:31.261690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:42601 (local address 127.8.18.195:32769)
0810 11:30:31.262592 (+ 902us) negotiation.cc:107] Waiting for socket to connect
0810 11:30:31.262640 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0810 11:30:31.264755 (+ 2115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:30:31.265880 (+ 1125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:30:31.265890 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:30:31.266604 (+ 714us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:30:31.267480 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:30:31.267511 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:30:31.269101 (+ 1590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:30:31.269105 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:30:31.270479 (+ 1374us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:30:31.270495 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:30:31.271421 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:30:31.271425 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:30:31.271644 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:31.272386 (+ 742us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:30:31.272409 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:30:31.275485 (+ 3076us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:30:31.279169 (+ 3684us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:30:31.279185 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:30:31.279204 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:30:31.279954 (+ 750us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:30:31.280498 (+ 544us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:30:31.280506 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:30:31.280510 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:30:31.280850 (+ 340us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:30:31.281651 (+ 801us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:30:31.281660 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:30:31.282091 (+ 431us) client_negotiation.cc:769] Sending connection context
0810 11:30:31.282375 (+ 284us) client_negotiation.cc:240] Negotiation successful
0810 11:30:31.282842 (+ 467us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":637,"thread_start_us":389,"threads_started":1}
I20260810 11:30:31.285907 11215 heartbeater.cc:347] Connected to a master server at 127.8.18.254:42601
I20260810 11:30:31.286278 11215 heartbeater.cc:464] Registering TS with master...
I20260810 11:30:31.287143 11215 heartbeater.cc:511] Master 127.8.18.254:42601 requested a full tablet report, sending...
I20260810 11:30:31.288929 10745 ts_manager.cc:194] Registered new tserver with Master: b5a9c24f60ae4326b4f0c0c210850815 (127.8.18.195:45777)
I20260810 11:30:31.290233 10745 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.18.195@KRBTEST.COM'} at 127.8.18.195:32769
I20260810 11:30:31.294731 8267 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260810 11:30:31.334609 11217 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:30:31.331741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55324 (local address 127.8.18.254:42601)
0810 11:30:31.331928 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0810 11:30:31.331932 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 11:30:31.332416 (+ 484us) server_negotiation.cc:407] Connection header received
0810 11:30:31.332577 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:30:31.332581 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:30:31.332672 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:30:31.332775 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:30:31.334209 (+ 1434us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55324: BlockingRecv error: recv got EOF from 127.0.0.1:55324 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
I20260810 11:30:31.338186 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 10808
W20260810 11:30:31.341713 8267 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 10808: Not authorized: Client connection negotiation failed: client connection to 127.8.18.193:34789: server requires authentication, but client does not have Kerberos credentials available
I20260810 11:30:31.341969 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 10808
I20260810 11:30:31.357648 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 10947
W20260810 11:30:31.360839 8267 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 10947: Not authorized: Client connection negotiation failed: client connection to 127.8.18.194:46415: server requires authentication, but client does not have Kerberos credentials available
I20260810 11:30:31.361052 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 10947
W20260810 11:30:31.361174 11228 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:30:31.358764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47710 (local address 127.8.18.194:46415)
0810 11:30:31.359403 (+ 639us) server_negotiation.cc:206] Beginning negotiation
0810 11:30:31.359410 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:30:31.359440 (+ 30us) server_negotiation.cc:407] Connection header received
0810 11:30:31.359504 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:30:31.359510 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:30:31.359755 (+ 245us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:30:31.359894 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:30:31.360768 (+ 874us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47710: BlockingRecv error: recv got EOF from 127.0.0.1:47710 (error 108)
Metrics: {"server-negotiator.queue_time_us":471,"thread_start_us":314,"threads_started":1}
I20260810 11:30:31.377641 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 11084
W20260810 11:30:31.380889 8267 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 11084: Not authorized: Client connection negotiation failed: client connection to 127.8.18.195:45777: server requires authentication, but client does not have Kerberos credentials available
I20260810 11:30:31.381059 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 11084
I20260810 11:30:31.399780 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 10713
I20260810 11:30:31.401255 10776 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:30:31.512259 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 10713
2026-08-10T11:30:31Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (3953 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786361401922362-8267-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 11:30:31 dist-test-slave-srgv krb5kdc[11234](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 11:30:31 dist-test-slave-srgv krb5kdc[11234](info): set up 2 sockets
Aug 10 11:30:31 dist-test-slave-srgv krb5kdc[11234](info): commencing operation
krb5kdc: starting...
W20260810 11:30:32.649310 8267 mini_kdc.cc:121] Time spent starting KDC: real 1.091s user 0.004s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786361401922362-8267-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786361401922362-8267-0/krb5kdc/test-user.keytab.
Aug 10 11:30:32 dist-test-slave-srgv krb5kdc[11234](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361432, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T11:30:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T11:30:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.18.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.18.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.18.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.254.keytab.
Entry for principal kudu/127.8.18.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.254.keytab.
I20260810 11:30:32.890012 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786361401922362-8267-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786361401922362-8267-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786361401922362-8267-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.18.254:41705
--webserver_interface=127.8.18.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.254.keytab
--principal=kudu/127.8.18.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.18.212:36463
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.18.254:41705
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786361401922362-8267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786361401922362-8267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786361401922362-8267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:30:33.247789 11247 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:30:33.248142 11247 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:30:33.248239 11247 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:30:33.256026 11247 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 11:30:33.256242 11247 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:30:33.256296 11247 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:30:33.256356 11247 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 11:30:33.256407 11247 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 11:30:33.267246 11247 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.18.212:36463
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786361401922362-8267-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786361401922362-8267-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.18.254:41705
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.254.keytab
--principal=kudu/127.8.18.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.18.254:41705
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.18.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786361401922362-8267-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:30:33.268700 11247 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:30:33.270232 11247 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:30:33.284301 11253 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:30:33.284322 11252 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:30:33.285226 11255 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:30:33.285372 11247 server_base.cc:1034] running on GCE node
I20260810 11:30:33.286402 11247 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:30:33.288054 11247 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:30:33.289319 11247 hybrid_clock.cc:648] HybridClock initialized: now 1786361433289246 us; error 64 us; skew 500 ppm
I20260810 11:30:33.289844 11247 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:30:33 dist-test-slave-srgv krb5kdc[11234](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361433, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:30:33.293887 11247 init.cc:377] Logged in from keytab as kudu/127.8.18.254@KRBTEST.COM (short username kudu)
I20260810 11:30:33.295941 11247 webserver.cc:460] Webserver started at http://127.8.18.254:38279/ using document root <none> and password file <none>
I20260810 11:30:33.296731 11247 fs_manager.cc:359] Metadata directory not provided
I20260810 11:30:33.296855 11247 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:30:33.297212 11247 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:30:33.300676 11247 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786361401922362-8267-0/minicluster-data/master-0/data/instance:
uuid: "b38ff11c468b469fb581f74bfd2536e9"
format_stamp: "Formatted at 2026-08-10 11:30:33 on dist-test-slave-srgv"
server_key: "4e4df4230b6bc1cc284dc0b3c9c71f8a"
server_key_iv: "1425aa1da04aa1a913cb67e42e237356"
server_key_version: "encryptionkey@0"
I20260810 11:30:33.301654 11247 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786361401922362-8267-0/minicluster-data/master-0/wal/instance:
uuid: "b38ff11c468b469fb581f74bfd2536e9"
format_stamp: "Formatted at 2026-08-10 11:30:33 on dist-test-slave-srgv"
server_key: "4e4df4230b6bc1cc284dc0b3c9c71f8a"
server_key_iv: "1425aa1da04aa1a913cb67e42e237356"
server_key_version: "encryptionkey@0"
I20260810 11:30:33.307852 11247 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260810 11:30:33.312031 11262 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:30:33.313923 11247 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260810 11:30:33.314201 11247 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786361401922362-8267-0/minicluster-data/master-0/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786361401922362-8267-0/minicluster-data/master-0/wal
uuid: "b38ff11c468b469fb581f74bfd2536e9"
format_stamp: "Formatted at 2026-08-10 11:30:33 on dist-test-slave-srgv"
server_key: "4e4df4230b6bc1cc284dc0b3c9c71f8a"
server_key_iv: "1425aa1da04aa1a913cb67e42e237356"
server_key_version: "encryptionkey@0"
I20260810 11:30:33.314451 11247 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786361401922362-8267-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786361401922362-8267-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786361401922362-8267-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:30:33.337015 11247 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:30:33.342723 11247 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:30:33.343130 11247 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:30:33.367630 11247 rpc_server.cc:307] RPC server started. Bound to: 127.8.18.254:41705
I20260810 11:30:33.367681 11315 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.18.254:41705 every 8 connection(s)
I20260810 11:30:33.370438 11247 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
I20260810 11:30:33.374920 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 11247
I20260810 11:30:33.375257 8267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786361401922362-8267-0/minicluster-data/master-0/wal/instance
I20260810 11:30:33.375902 11316 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:30:33.376163 8267 external_mini_cluster.cc:1428] Setting key 6467de092141ebe60267ea99e3ed35a0
I20260810 11:30:33.393329 11316 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b38ff11c468b469fb581f74bfd2536e9: Bootstrap starting.
Aug 10 11:30:33 dist-test-slave-srgv krb5kdc[11234](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361432, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.18.254@KRBTEST.COM
I20260810 11:30:33.397207 11316 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b38ff11c468b469fb581f74bfd2536e9: Neither blocks nor log segments found. Creating new log.
I20260810 11:30:33.398629 11316 log.cc:824] T 00000000000000000000000000000000 P b38ff11c468b469fb581f74bfd2536e9: Log is configured to *not* fsync() on all Append() calls
I20260810 11:30:33.402107 11316 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b38ff11c468b469fb581f74bfd2536e9: No bootstrap required, opened a new log
I20260810 11:30:33.403815 11319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:33.378601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37448 (local address 127.8.18.254:41705)
0810 11:30:33.379601 (+ 1000us) server_negotiation.cc:206] Beginning negotiation
0810 11:30:33.379617 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0810 11:30:33.379669 (+ 52us) server_negotiation.cc:407] Connection header received
0810 11:30:33.381761 (+ 2092us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:30:33.381795 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:30:33.382352 (+ 557us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:30:33.382779 (+ 427us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:30:33.384959 (+ 2180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:33.386470 (+ 1511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:33.388463 (+ 1993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:33.389313 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:33.389604 (+ 291us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:33.395027 (+ 5423us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:30:33.395037 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:30:33.395101 (+ 64us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:30:33.395196 (+ 95us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:30:33.398869 (+ 3673us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:33.399852 (+ 983us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:33.399898 (+ 46us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:33.399907 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:33.400122 (+ 215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:33.400637 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:33.400643 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:33.400649 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:33.401277 (+ 628us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:30:33.401657 (+ 380us) server_negotiation.cc:1035] Waiting for connection context
0810 11:30:33.402192 (+ 535us) server_negotiation.cc:299] Negotiation successful
0810 11:30:33.402739 (+ 547us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":735,"thread_start_us":469,"threads_started":1}
I20260810 11:30:33.408754 8267 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 11:30:33.411309 11316 raft_consensus.cc:348] T 00000000000000000000000000000000 P b38ff11c468b469fb581f74bfd2536e9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b38ff11c468b469fb581f74bfd2536e9" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 41705 } }
I20260810 11:30:33.411799 11316 raft_consensus.cc:374] T 00000000000000000000000000000000 P b38ff11c468b469fb581f74bfd2536e9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:30:33.411918 11316 raft_consensus.cc:729] T 00000000000000000000000000000000 P b38ff11c468b469fb581f74bfd2536e9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b38ff11c468b469fb581f74bfd2536e9, State: Initialized, Role: FOLLOWER
I20260810 11:30:33.412585 11316 consensus_queue.cc:260] T 00000000000000000000000000000000 P b38ff11c468b469fb581f74bfd2536e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b38ff11c468b469fb581f74bfd2536e9" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 41705 } }
I20260810 11:30:33.412881 11316 raft_consensus.cc:388] T 00000000000000000000000000000000 P b38ff11c468b469fb581f74bfd2536e9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 11:30:33.413040 11316 raft_consensus.cc:482] T 00000000000000000000000000000000 P b38ff11c468b469fb581f74bfd2536e9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 11:30:33.413237 11316 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b38ff11c468b469fb581f74bfd2536e9 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:30:33.415788 11316 raft_consensus.cc:504] T 00000000000000000000000000000000 P b38ff11c468b469fb581f74bfd2536e9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b38ff11c468b469fb581f74bfd2536e9" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 41705 } }
I20260810 11:30:33.416594 11316 leader_election.cc:302] T 00000000000000000000000000000000 P b38ff11c468b469fb581f74bfd2536e9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b38ff11c468b469fb581f74bfd2536e9; no voters:
I20260810 11:30:33.417469 11316 leader_election.cc:288] T 00000000000000000000000000000000 P b38ff11c468b469fb581f74bfd2536e9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 11:30:33.417651 11322 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b38ff11c468b469fb581f74bfd2536e9 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:30:33.418363 11322 raft_consensus.cc:686] T 00000000000000000000000000000000 P b38ff11c468b469fb581f74bfd2536e9 [term 1 LEADER]: Becoming Leader. State: Replica: b38ff11c468b469fb581f74bfd2536e9, State: Running, Role: LEADER
I20260810 11:30:33.419102 11322 consensus_queue.cc:237] T 00000000000000000000000000000000 P b38ff11c468b469fb581f74bfd2536e9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b38ff11c468b469fb581f74bfd2536e9" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 41705 } }
I20260810 11:30:33.421346 11322 sys_catalog.cc:455] T 00000000000000000000000000000000 P b38ff11c468b469fb581f74bfd2536e9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b38ff11c468b469fb581f74bfd2536e9. Latest consensus state: current_term: 1 leader_uuid: "b38ff11c468b469fb581f74bfd2536e9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b38ff11c468b469fb581f74bfd2536e9" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 41705 } } }
I20260810 11:30:33.421749 11322 sys_catalog.cc:458] T 00000000000000000000000000000000 P b38ff11c468b469fb581f74bfd2536e9 [sys.catalog]: This master's current role is: LEADER
I20260810 11:30:33.422289 11324 sys_catalog.cc:455] T 00000000000000000000000000000000 P b38ff11c468b469fb581f74bfd2536e9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b38ff11c468b469fb581f74bfd2536e9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b38ff11c468b469fb581f74bfd2536e9" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 41705 } } }
I20260810 11:30:33.422678 11324 sys_catalog.cc:458] T 00000000000000000000000000000000 P b38ff11c468b469fb581f74bfd2536e9 [sys.catalog]: This master's current role is: LEADER
I20260810 11:30:33.426282 11316 sys_catalog.cc:564] T 00000000000000000000000000000000 P b38ff11c468b469fb581f74bfd2536e9 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 11:30:33.430197 11326 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 11:30:33.440519 11326 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 11:30:33.466495 11326 catalog_manager.cc:1347] Generated new cluster ID: 2cbf495e44d14bf1955c82b9db6800f2
I20260810 11:30:33.466681 11326 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 11:30:33.496416 11343 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260810 11:30:33.508574 11326 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 11:30:33.510497 11319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:33.499753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37458 (local address 127.8.18.254:41705)
0810 11:30:33.499973 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0810 11:30:33.499980 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:30:33.500659 (+ 679us) server_negotiation.cc:407] Connection header received
0810 11:30:33.500878 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:30:33.500884 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:30:33.500974 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:30:33.501089 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:30:33.502614 (+ 1525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:33.503814 (+ 1200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:33.506005 (+ 2191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:33.506705 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:33.506868 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:33.508332 (+ 1464us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:30:33.508338 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:30:33.508343 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:30:33.509499 (+ 1156us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:30:33.509570 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:30:33.509610 (+ 40us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:30:33.509789 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0810 11:30:33.509946 (+ 157us) server_negotiation.cc:299] Negotiation successful
0810 11:30:33.510062 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":76}
I20260810 11:30:33.510723 11326 catalog_manager.cc:1504] Loading token signing keys...
I20260810 11:30:33.520087 11326 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b38ff11c468b469fb581f74bfd2536e9: Generated new TSK 0
I20260810 11:30:33.521239 11326 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 11:30:33.532582 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 11247
I20260810 11:30:33.534725 11310 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:30:33.632858 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 11247
2026-08-10T11:30:33Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (2115 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786361401922362-8267-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 11:30:33 dist-test-slave-srgv krb5kdc[11350](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 11:30:33 dist-test-slave-srgv krb5kdc[11350](info): set up 2 sockets
Aug 10 11:30:33 dist-test-slave-srgv krb5kdc[11350](info): commencing operation
krb5kdc: starting...
W20260810 11:30:34.768965 8267 mini_kdc.cc:121] Time spent starting KDC: real 1.096s user 0.004s sys 0.020s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786361401922362-8267-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786361401922362-8267-0/krb5kdc/test-user.keytab.
Aug 10 11:30:34 dist-test-slave-srgv krb5kdc[11350](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361434, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T11:30:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T11:30:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.18.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.18.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.18.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.254.keytab.
Entry for principal kudu/127.8.18.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.254.keytab.
I20260810 11:30:35.010944 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786361401922362-8267-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786361401922362-8267-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786361401922362-8267-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.18.254:41853
--webserver_interface=127.8.18.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.254.keytab
--principal=kudu/127.8.18.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.18.212:42417
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.18.254:41853
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786361401922362-8267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786361401922362-8267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786361401922362-8267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:30:35.367588 11363 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:30:35.368012 11363 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:30:35.368187 11363 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:30:35.376066 11363 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 11:30:35.376287 11363 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:30:35.376396 11363 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:30:35.376511 11363 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 11:30:35.376645 11363 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 11:30:35.387454 11363 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.18.212:42417
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786361401922362-8267-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786361401922362-8267-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.18.254:41853
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.254.keytab
--principal=kudu/127.8.18.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.18.254:41853
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.18.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786361401922362-8267-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:30:35.389024 11363 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:30:35.390605 11363 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:30:35.403949 11368 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:30:35.404065 11371 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:30:35.404223 11369 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:30:35.405360 11363 server_base.cc:1034] running on GCE node
I20260810 11:30:35.406423 11363 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:30:35.408106 11363 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:30:35.409507 11363 hybrid_clock.cc:648] HybridClock initialized: now 1786361435409357 us; error 59 us; skew 500 ppm
I20260810 11:30:35.409961 11363 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:30:35 dist-test-slave-srgv krb5kdc[11350](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361435, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:30:35.419677 11363 init.cc:377] Logged in from keytab as kudu/127.8.18.254@KRBTEST.COM (short username kudu)
I20260810 11:30:35.421808 11363 webserver.cc:460] Webserver started at http://127.8.18.254:39133/ using document root <none> and password file <none>
I20260810 11:30:35.422673 11363 fs_manager.cc:359] Metadata directory not provided
I20260810 11:30:35.422849 11363 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:30:35.423202 11363 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:30:35.426707 11363 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786361401922362-8267-0/minicluster-data/master-0/data/instance:
uuid: "30cadc01f5a84f129f417dc9099f9fe8"
format_stamp: "Formatted at 2026-08-10 11:30:35 on dist-test-slave-srgv"
server_key: "8cfb8846b0f528adb533b9f23ee347f7"
server_key_iv: "6428897c46f2cfb95bfbfb825fd0ac65"
server_key_version: "encryptionkey@0"
I20260810 11:30:35.427591 11363 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786361401922362-8267-0/minicluster-data/master-0/wal/instance:
uuid: "30cadc01f5a84f129f417dc9099f9fe8"
format_stamp: "Formatted at 2026-08-10 11:30:35 on dist-test-slave-srgv"
server_key: "8cfb8846b0f528adb533b9f23ee347f7"
server_key_iv: "6428897c46f2cfb95bfbfb825fd0ac65"
server_key_version: "encryptionkey@0"
I20260810 11:30:35.433787 11363 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260810 11:30:35.438102 11379 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:30:35.439877 11363 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260810 11:30:35.440161 11363 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786361401922362-8267-0/minicluster-data/master-0/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786361401922362-8267-0/minicluster-data/master-0/wal
uuid: "30cadc01f5a84f129f417dc9099f9fe8"
format_stamp: "Formatted at 2026-08-10 11:30:35 on dist-test-slave-srgv"
server_key: "8cfb8846b0f528adb533b9f23ee347f7"
server_key_iv: "6428897c46f2cfb95bfbfb825fd0ac65"
server_key_version: "encryptionkey@0"
I20260810 11:30:35.440428 11363 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786361401922362-8267-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786361401922362-8267-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786361401922362-8267-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:30:35.456336 11363 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:30:35.461484 11363 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:30:35.461858 11363 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:30:35.483784 11363 rpc_server.cc:307] RPC server started. Bound to: 127.8.18.254:41853
I20260810 11:30:35.483820 11430 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.18.254:41853 every 8 connection(s)
I20260810 11:30:35.485508 11363 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
I20260810 11:30:35.489704 11431 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:30:35.495823 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 11363
I20260810 11:30:35.496095 8267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786361401922362-8267-0/minicluster-data/master-0/wal/instance
I20260810 11:30:35.496961 8267 external_mini_cluster.cc:1428] Setting key a6d1a26c9adf02879f1993d814c96ddd
I20260810 11:30:35.502719 11431 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 30cadc01f5a84f129f417dc9099f9fe8: Bootstrap starting.
I20260810 11:30:35.507081 11431 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 30cadc01f5a84f129f417dc9099f9fe8: Neither blocks nor log segments found. Creating new log.
I20260810 11:30:35.508700 11431 log.cc:824] T 00000000000000000000000000000000 P 30cadc01f5a84f129f417dc9099f9fe8: Log is configured to *not* fsync() on all Append() calls
I20260810 11:30:35.513090 11431 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 30cadc01f5a84f129f417dc9099f9fe8: No bootstrap required, opened a new log
Aug 10 11:30:35 dist-test-slave-srgv krb5kdc[11350](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361434, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.18.254@KRBTEST.COM
I20260810 11:30:35.522137 11431 raft_consensus.cc:348] T 00000000000000000000000000000000 P 30cadc01f5a84f129f417dc9099f9fe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30cadc01f5a84f129f417dc9099f9fe8" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 41853 } }
I20260810 11:30:35.522639 11431 raft_consensus.cc:374] T 00000000000000000000000000000000 P 30cadc01f5a84f129f417dc9099f9fe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:30:35.522795 11431 raft_consensus.cc:729] T 00000000000000000000000000000000 P 30cadc01f5a84f129f417dc9099f9fe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30cadc01f5a84f129f417dc9099f9fe8, State: Initialized, Role: FOLLOWER
I20260810 11:30:35.523654 11431 consensus_queue.cc:260] T 00000000000000000000000000000000 P 30cadc01f5a84f129f417dc9099f9fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30cadc01f5a84f129f417dc9099f9fe8" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 41853 } }
I20260810 11:30:35.523985 11431 raft_consensus.cc:388] T 00000000000000000000000000000000 P 30cadc01f5a84f129f417dc9099f9fe8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 11:30:35.524205 11431 raft_consensus.cc:482] T 00000000000000000000000000000000 P 30cadc01f5a84f129f417dc9099f9fe8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 11:30:35.524426 11431 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 30cadc01f5a84f129f417dc9099f9fe8 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:30:35.525393 11434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:35.499472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52852 (local address 127.8.18.254:41853)
0810 11:30:35.500489 (+ 1017us) server_negotiation.cc:206] Beginning negotiation
0810 11:30:35.500502 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 11:30:35.500607 (+ 105us) server_negotiation.cc:407] Connection header received
0810 11:30:35.502587 (+ 1980us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:30:35.502614 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:30:35.503190 (+ 576us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:30:35.503589 (+ 399us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:30:35.505790 (+ 2201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:35.507391 (+ 1601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:35.509408 (+ 2017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:35.510816 (+ 1408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:35.511004 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:35.516473 (+ 5469us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:30:35.516483 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:30:35.516511 (+ 28us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:30:35.516570 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:30:35.520256 (+ 3686us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:35.521207 (+ 951us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:35.521215 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:35.521224 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:35.521496 (+ 272us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:35.521967 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:35.521973 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:35.521976 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:35.522716 (+ 740us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:30:35.523070 (+ 354us) server_negotiation.cc:1035] Waiting for connection context
0810 11:30:35.523582 (+ 512us) server_negotiation.cc:299] Negotiation successful
0810 11:30:35.524091 (+ 509us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":761,"thread_start_us":503,"threads_started":1}
I20260810 11:30:35.527909 11431 raft_consensus.cc:504] T 00000000000000000000000000000000 P 30cadc01f5a84f129f417dc9099f9fe8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30cadc01f5a84f129f417dc9099f9fe8" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 41853 } }
I20260810 11:30:35.528492 11431 leader_election.cc:302] T 00000000000000000000000000000000 P 30cadc01f5a84f129f417dc9099f9fe8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 30cadc01f5a84f129f417dc9099f9fe8; no voters:
I20260810 11:30:35.529323 11431 leader_election.cc:288] T 00000000000000000000000000000000 P 30cadc01f5a84f129f417dc9099f9fe8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 11:30:35.529397 11436 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 30cadc01f5a84f129f417dc9099f9fe8 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:30:35.530432 11436 raft_consensus.cc:686] T 00000000000000000000000000000000 P 30cadc01f5a84f129f417dc9099f9fe8 [term 1 LEADER]: Becoming Leader. State: Replica: 30cadc01f5a84f129f417dc9099f9fe8, State: Running, Role: LEADER
I20260810 11:30:35.531121 11436 consensus_queue.cc:237] T 00000000000000000000000000000000 P 30cadc01f5a84f129f417dc9099f9fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30cadc01f5a84f129f417dc9099f9fe8" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 41853 } }
I20260810 11:30:35.531952 11431 sys_catalog.cc:564] T 00000000000000000000000000000000 P 30cadc01f5a84f129f417dc9099f9fe8 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 11:30:35.534600 11437 sys_catalog.cc:455] T 00000000000000000000000000000000 P 30cadc01f5a84f129f417dc9099f9fe8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "30cadc01f5a84f129f417dc9099f9fe8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30cadc01f5a84f129f417dc9099f9fe8" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 41853 } } }
I20260810 11:30:35.534704 11438 sys_catalog.cc:455] T 00000000000000000000000000000000 P 30cadc01f5a84f129f417dc9099f9fe8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 30cadc01f5a84f129f417dc9099f9fe8. Latest consensus state: current_term: 1 leader_uuid: "30cadc01f5a84f129f417dc9099f9fe8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30cadc01f5a84f129f417dc9099f9fe8" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 41853 } } }
I20260810 11:30:35.535110 11437 sys_catalog.cc:458] T 00000000000000000000000000000000 P 30cadc01f5a84f129f417dc9099f9fe8 [sys.catalog]: This master's current role is: LEADER
I20260810 11:30:35.535166 11438 sys_catalog.cc:458] T 00000000000000000000000000000000 P 30cadc01f5a84f129f417dc9099f9fe8 [sys.catalog]: This master's current role is: LEADER
I20260810 11:30:35.537686 11443 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 11:30:35.543740 11443 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 11:30:35.555251 11443 catalog_manager.cc:1347] Generated new cluster ID: 638337d32c10482ea23b576807fb309a
I20260810 11:30:35.555408 11443 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 11:30:35.570258 11443 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 11:30:35.572413 11443 catalog_manager.cc:1504] Loading token signing keys...
I20260810 11:30:35.583851 11443 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 30cadc01f5a84f129f417dc9099f9fe8: Generated new TSK 0
I20260810 11:30:35.585161 11443 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.18.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.18.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.18.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.193.keytab.
Entry for principal kudu/127.8.18.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.193.keytab.
I20260810 11:30:35.645959 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786361401922362-8267-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786361401922362-8267-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786361401922362-8267-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786361401922362-8267-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.18.193:0
--local_ip_for_outbound_sockets=127.8.18.193
--webserver_interface=127.8.18.193
--webserver_port=0
--tserver_master_addrs=127.8.18.254:41853
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.193.keytab
--principal=kudu/127.8.18.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.18.212:42417
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786361401922362-8267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786361401922362-8267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786361401922362-8267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:30:36.011011 11458 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:30:36.011345 11458 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:30:36.011508 11458 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:30:36.019294 11458 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:30:36.019498 11458 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:30:36.019703 11458 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.18.193
I20260810 11:30:36.030275 11458 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.18.212:42417
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786361401922362-8267-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786361401922362-8267-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.193.keytab
--principal=kudu/127.8.18.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.18.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786361401922362-8267-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.18.193
--webserver_port=0
--tserver_master_addrs=127.8.18.254:41853
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.18.193
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786361401922362-8267-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:30:36.031915 11458 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:30:36.033730 11458 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:30:36.049127 11463 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:30:36.049207 11464 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:30:36.049257 11466 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:30:36.050371 11458 server_base.cc:1034] running on GCE node
I20260810 11:30:36.051079 11458 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:30:36.052127 11458 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:30:36.053473 11458 hybrid_clock.cc:648] HybridClock initialized: now 1786361436053344 us; error 36 us; skew 500 ppm
I20260810 11:30:36.053889 11458 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:30:36 dist-test-slave-srgv krb5kdc[11350](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361436, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:30:36.057654 11458 init.cc:377] Logged in from keytab as kudu/127.8.18.193@KRBTEST.COM (short username kudu)
I20260810 11:30:36.059430 11458 webserver.cc:460] Webserver started at http://127.8.18.193:33485/ using document root <none> and password file <none>
I20260810 11:30:36.060240 11458 fs_manager.cc:359] Metadata directory not provided
I20260810 11:30:36.060364 11458 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:30:36.060732 11458 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:30:36.064340 11458 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786361401922362-8267-0/minicluster-data/ts-0/data/instance:
uuid: "9314c7e6871d4f259d3c10a08c2cd7b9"
format_stamp: "Formatted at 2026-08-10 11:30:36 on dist-test-slave-srgv"
server_key: "dcf9a48caa65b403cd1f489e7aab80fe"
server_key_iv: "7b0d4ef38db57bff1c12b88423e6c0c5"
server_key_version: "encryptionkey@0"
I20260810 11:30:36.065364 11458 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786361401922362-8267-0/minicluster-data/ts-0/wal/instance:
uuid: "9314c7e6871d4f259d3c10a08c2cd7b9"
format_stamp: "Formatted at 2026-08-10 11:30:36 on dist-test-slave-srgv"
server_key: "dcf9a48caa65b403cd1f489e7aab80fe"
server_key_iv: "7b0d4ef38db57bff1c12b88423e6c0c5"
server_key_version: "encryptionkey@0"
I20260810 11:30:36.072386 11458 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.001s sys 0.004s
I20260810 11:30:36.077066 11473 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:30:36.078999 11458 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260810 11:30:36.079288 11458 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786361401922362-8267-0/minicluster-data/ts-0/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786361401922362-8267-0/minicluster-data/ts-0/wal
uuid: "9314c7e6871d4f259d3c10a08c2cd7b9"
format_stamp: "Formatted at 2026-08-10 11:30:36 on dist-test-slave-srgv"
server_key: "dcf9a48caa65b403cd1f489e7aab80fe"
server_key_iv: "7b0d4ef38db57bff1c12b88423e6c0c5"
server_key_version: "encryptionkey@0"
I20260810 11:30:36.079547 11458 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786361401922362-8267-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786361401922362-8267-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786361401922362-8267-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:30:36.099298 11458 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:30:36.104113 11458 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:30:36.104453 11458 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:30:36.105628 11458 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:30:36.107170 11458 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:30:36.107293 11458 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:30:36.107458 11458 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:30:36.107534 11458 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:30:36.145071 11458 rpc_server.cc:307] RPC server started. Bound to: 127.8.18.193:42819
I20260810 11:30:36.145119 11590 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.18.193:42819 every 8 connection(s)
I20260810 11:30:36.146704 11458 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786361401922362-8267-0/minicluster-data/ts-0/data/info.pb
I20260810 11:30:36.152926 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 11458
I20260810 11:30:36.153227 8267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786361401922362-8267-0/minicluster-data/ts-0/wal/instance
I20260810 11:30:36.154276 8267 external_mini_cluster.cc:1428] Setting key f6d38ea6804f9e29e73562b45081aad4
Aug 10 11:30:36 dist-test-slave-srgv krb5kdc[11350](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361436, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.193@KRBTEST.COM for kudu/127.8.18.254@KRBTEST.COM
I20260810 11:30:36.175459 11594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:36.149674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.193:39095 (local address 127.8.18.254:41853)
0810 11:30:36.150155 (+ 481us) server_negotiation.cc:206] Beginning negotiation
0810 11:30:36.150159 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 11:30:36.151741 (+ 1582us) server_negotiation.cc:407] Connection header received
0810 11:30:36.154667 (+ 2926us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:30:36.154674 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:30:36.154867 (+ 193us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:30:36.155005 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:30:36.157919 (+ 2914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:36.159229 (+ 1310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:36.161318 (+ 2089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:36.162063 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:36.162244 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:36.167004 (+ 4760us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:30:36.167060 (+ 56us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:30:36.167063 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:30:36.167116 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:30:36.170632 (+ 3516us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:36.172051 (+ 1419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:36.172058 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:36.172060 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:36.172278 (+ 218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:36.172939 (+ 661us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:36.172943 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:36.172945 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:36.173361 (+ 416us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:30:36.173586 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0810 11:30:36.174817 (+ 1231us) server_negotiation.cc:299] Negotiation successful
0810 11:30:36.175109 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":370,"thread_start_us":245,"threads_started":1}
I20260810 11:30:36.177166 11595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:36.150315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:41853 (local address 127.8.18.193:39095)
0810 11:30:36.151248 (+ 933us) negotiation.cc:107] Waiting for socket to connect
0810 11:30:36.151302 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0810 11:30:36.154218 (+ 2916us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:30:36.155696 (+ 1478us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:30:36.155722 (+ 26us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:30:36.156653 (+ 931us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:30:36.157668 (+ 1015us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:30:36.157702 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:30:36.159540 (+ 1838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:30:36.159545 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:30:36.161060 (+ 1515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:30:36.161077 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:30:36.162377 (+ 1300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:30:36.162386 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:30:36.162681 (+ 295us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:36.163592 (+ 911us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:30:36.163619 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:30:36.166631 (+ 3012us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:30:36.170911 (+ 4280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:30:36.170926 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:30:36.170951 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:30:36.171760 (+ 809us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:30:36.172446 (+ 686us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:30:36.172453 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:30:36.172460 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:30:36.172797 (+ 337us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:30:36.173597 (+ 800us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:30:36.173610 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:30:36.174184 (+ 574us) client_negotiation.cc:769] Sending connection context
0810 11:30:36.174576 (+ 392us) client_negotiation.cc:240] Negotiation successful
0810 11:30:36.175342 (+ 766us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":539,"thread_start_us":336,"threads_started":1}
I20260810 11:30:36.179303 11591 heartbeater.cc:347] Connected to a master server at 127.8.18.254:41853
WARNING: no policy specified for kudu/127.8.18.194@KRBTEST.COM; defaulting to no policy
I20260810 11:30:36.179713 11591 heartbeater.cc:464] Registering TS with master...
I20260810 11:30:36.180696 11591 heartbeater.cc:511] Master 127.8.18.254:41853 requested a full tablet report, sending...
I20260810 11:30:36.183364 11396 ts_manager.cc:194] Registered new tserver with Master: 9314c7e6871d4f259d3c10a08c2cd7b9 (127.8.18.193:42819)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.18.194@KRBTEST.COM" created.
I20260810 11:30:36.185815 11396 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.18.193@KRBTEST.COM'} at 127.8.18.193:39095
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.18.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.194.keytab.
Entry for principal kudu/127.8.18.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.194.keytab.
I20260810 11:30:36.214617 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786361401922362-8267-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786361401922362-8267-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786361401922362-8267-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786361401922362-8267-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.18.194:0
--local_ip_for_outbound_sockets=127.8.18.194
--webserver_interface=127.8.18.194
--webserver_port=0
--tserver_master_addrs=127.8.18.254:41853
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.194.keytab
--principal=kudu/127.8.18.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.18.212:42417
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786361401922362-8267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786361401922362-8267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786361401922362-8267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:30:36.599027 11598 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:30:36.599370 11598 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:30:36.599470 11598 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:30:36.607321 11598 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:30:36.607465 11598 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:30:36.607651 11598 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.18.194
I20260810 11:30:36.618716 11598 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.18.212:42417
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786361401922362-8267-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786361401922362-8267-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.194.keytab
--principal=kudu/127.8.18.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.18.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786361401922362-8267-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.18.194
--webserver_port=0
--tserver_master_addrs=127.8.18.254:41853
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.18.194
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786361401922362-8267-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:30:36.620275 11598 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:30:36.621801 11598 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:30:36.636847 11603 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:30:36.636833 11604 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:30:36.637169 11606 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:30:36.638202 11598 server_base.cc:1034] running on GCE node
I20260810 11:30:36.638973 11598 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:30:36.640123 11598 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:30:36.641475 11598 hybrid_clock.cc:648] HybridClock initialized: now 1786361436641351 us; error 63 us; skew 500 ppm
I20260810 11:30:36.641875 11598 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:30:36 dist-test-slave-srgv krb5kdc[11350](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361436, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:30:36.645958 11598 init.cc:377] Logged in from keytab as kudu/127.8.18.194@KRBTEST.COM (short username kudu)
I20260810 11:30:36.647902 11598 webserver.cc:460] Webserver started at http://127.8.18.194:38905/ using document root <none> and password file <none>
I20260810 11:30:36.648917 11598 fs_manager.cc:359] Metadata directory not provided
I20260810 11:30:36.649096 11598 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:30:36.649528 11598 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:30:36.652833 11598 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786361401922362-8267-0/minicluster-data/ts-1/data/instance:
uuid: "d35c6c34dee74dde95d76fe1c1848e7a"
format_stamp: "Formatted at 2026-08-10 11:30:36 on dist-test-slave-srgv"
server_key: "4df1b5b95bdb614722d12b3c0cc5f752"
server_key_iv: "250f1efaaf5cab1d876a4c990b90c973"
server_key_version: "encryptionkey@0"
I20260810 11:30:36.653825 11598 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786361401922362-8267-0/minicluster-data/ts-1/wal/instance:
uuid: "d35c6c34dee74dde95d76fe1c1848e7a"
format_stamp: "Formatted at 2026-08-10 11:30:36 on dist-test-slave-srgv"
server_key: "4df1b5b95bdb614722d12b3c0cc5f752"
server_key_iv: "250f1efaaf5cab1d876a4c990b90c973"
server_key_version: "encryptionkey@0"
I20260810 11:30:36.660117 11598 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.001s
I20260810 11:30:36.664400 11613 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:30:36.666283 11598 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260810 11:30:36.666525 11598 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786361401922362-8267-0/minicluster-data/ts-1/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786361401922362-8267-0/minicluster-data/ts-1/wal
uuid: "d35c6c34dee74dde95d76fe1c1848e7a"
format_stamp: "Formatted at 2026-08-10 11:30:36 on dist-test-slave-srgv"
server_key: "4df1b5b95bdb614722d12b3c0cc5f752"
server_key_iv: "250f1efaaf5cab1d876a4c990b90c973"
server_key_version: "encryptionkey@0"
I20260810 11:30:36.666766 11598 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786361401922362-8267-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786361401922362-8267-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786361401922362-8267-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:30:36.689911 11598 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:30:36.695494 11598 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:30:36.695890 11598 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:30:36.697031 11598 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:30:36.698596 11598 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:30:36.698721 11598 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:30:36.698892 11598 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:30:36.698971 11598 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:30:36.735291 11598 rpc_server.cc:307] RPC server started. Bound to: 127.8.18.194:35929
I20260810 11:30:36.735395 11729 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.18.194:35929 every 8 connection(s)
I20260810 11:30:36.736790 11598 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786361401922362-8267-0/minicluster-data/ts-1/data/info.pb
I20260810 11:30:36.740778 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 11598
I20260810 11:30:36.741042 8267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786361401922362-8267-0/minicluster-data/ts-1/wal/instance
I20260810 11:30:36.741642 8267 external_mini_cluster.cc:1428] Setting key 67db9f9371f14b6d08fb011626efdd78
Aug 10 11:30:36 dist-test-slave-srgv krb5kdc[11350](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361436, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.194@KRBTEST.COM for kudu/127.8.18.254@KRBTEST.COM
I20260810 11:30:36.762818 11732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:36.740068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.194:36779 (local address 127.8.18.254:41853)
0810 11:30:36.740722 (+ 654us) server_negotiation.cc:206] Beginning negotiation
0810 11:30:36.740730 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:30:36.742101 (+ 1371us) server_negotiation.cc:407] Connection header received
0810 11:30:36.744358 (+ 2257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:30:36.744364 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:30:36.744464 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:30:36.744623 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:30:36.746790 (+ 2167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:36.748075 (+ 1285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:36.749958 (+ 1883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:36.750776 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:36.750962 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:36.754976 (+ 4014us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:30:36.754982 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:30:36.754985 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:30:36.755025 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:30:36.758278 (+ 3253us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:36.759296 (+ 1018us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:36.759301 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:36.759304 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:36.759512 (+ 208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:36.760299 (+ 787us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:36.760304 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:36.760306 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:36.760618 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:30:36.760767 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0810 11:30:36.762065 (+ 1298us) server_negotiation.cc:299] Negotiation successful
0810 11:30:36.762293 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":503,"thread_start_us":329,"threads_started":1}
I20260810 11:30:36.764468 11733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:36.740833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:41853 (local address 127.8.18.194:36779)
0810 11:30:36.741896 (+ 1063us) negotiation.cc:107] Waiting for socket to connect
0810 11:30:36.741946 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0810 11:30:36.744036 (+ 2090us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:30:36.745040 (+ 1004us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:30:36.745051 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:30:36.745893 (+ 842us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:30:36.746556 (+ 663us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:30:36.746578 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:30:36.748270 (+ 1692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:30:36.748274 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:30:36.749775 (+ 1501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:30:36.749791 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:30:36.751087 (+ 1296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:30:36.751096 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:30:36.751368 (+ 272us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:36.752172 (+ 804us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:30:36.752202 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:30:36.754655 (+ 2453us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:30:36.758492 (+ 3837us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:30:36.758504 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:30:36.758532 (+ 28us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:30:36.759116 (+ 584us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:30:36.759711 (+ 595us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:30:36.759720 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:30:36.759724 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:30:36.760051 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:30:36.760821 (+ 770us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:30:36.760833 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:30:36.761515 (+ 682us) client_negotiation.cc:769] Sending connection context
0810 11:30:36.761958 (+ 443us) client_negotiation.cc:240] Negotiation successful
0810 11:30:36.762710 (+ 752us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":755,"thread_start_us":449,"threads_started":1}
WARNING: no policy specified for kudu/127.8.18.195@KRBTEST.COM; defaulting to no policy
I20260810 11:30:36.766438 11730 heartbeater.cc:347] Connected to a master server at 127.8.18.254:41853
I20260810 11:30:36.766826 11730 heartbeater.cc:464] Registering TS with master...
I20260810 11:30:36.767850 11730 heartbeater.cc:511] Master 127.8.18.254:41853 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.18.195@KRBTEST.COM" created.
I20260810 11:30:36.769949 11396 ts_manager.cc:194] Registered new tserver with Master: d35c6c34dee74dde95d76fe1c1848e7a (127.8.18.194:35929)
I20260810 11:30:36.771008 11396 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.18.194@KRBTEST.COM'} at 127.8.18.194:36779
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.18.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.195.keytab.
Entry for principal kudu/127.8.18.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.195.keytab.
I20260810 11:30:36.797674 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786361401922362-8267-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786361401922362-8267-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786361401922362-8267-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786361401922362-8267-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.18.195:0
--local_ip_for_outbound_sockets=127.8.18.195
--webserver_interface=127.8.18.195
--webserver_port=0
--tserver_master_addrs=127.8.18.254:41853
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.195.keytab
--principal=kudu/127.8.18.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.18.212:42417
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786361401922362-8267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786361401922362-8267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786361401922362-8267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:30:37.153982 11736 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:30:37.154299 11736 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:30:37.154397 11736 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:30:37.161919 11736 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:30:37.162055 11736 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:30:37.162171 11736 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.18.195
I20260810 11:30:37.172884 11736 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.18.212:42417
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786361401922362-8267-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786361401922362-8267-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.195.keytab
--principal=kudu/127.8.18.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.18.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786361401922362-8267-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.18.195
--webserver_port=0
--tserver_master_addrs=127.8.18.254:41853
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.18.195
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786361401922362-8267-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:30:37.174348 11736 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:30:37.176098 11736 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 11:30:37.189711 11591 heartbeater.cc:503] Master 127.8.18.254:41853 was elected leader, sending a full tablet report...
W20260810 11:30:37.190218 11741 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:30:37.190332 11744 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:30:37.191035 11742 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:30:37.191429 11736 server_base.cc:1034] running on GCE node
I20260810 11:30:37.192227 11736 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:30:37.193269 11736 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:30:37.194554 11736 hybrid_clock.cc:648] HybridClock initialized: now 1786361437194492 us; error 76 us; skew 500 ppm
I20260810 11:30:37.194962 11736 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:30:37 dist-test-slave-srgv krb5kdc[11350](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361437, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:30:37.198704 11736 init.cc:377] Logged in from keytab as kudu/127.8.18.195@KRBTEST.COM (short username kudu)
I20260810 11:30:37.200685 11736 webserver.cc:460] Webserver started at http://127.8.18.195:33877/ using document root <none> and password file <none>
I20260810 11:30:37.201789 11736 fs_manager.cc:359] Metadata directory not provided
I20260810 11:30:37.201969 11736 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:30:37.202401 11736 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:30:37.206511 11736 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786361401922362-8267-0/minicluster-data/ts-2/data/instance:
uuid: "de988b5a93624b1e84383509a204a836"
format_stamp: "Formatted at 2026-08-10 11:30:37 on dist-test-slave-srgv"
server_key: "1ff04f0915b8373ae8755b6f4fb6ea04"
server_key_iv: "5093088c219b6c2f6185fef22ff6305c"
server_key_version: "encryptionkey@0"
I20260810 11:30:37.207427 11736 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786361401922362-8267-0/minicluster-data/ts-2/wal/instance:
uuid: "de988b5a93624b1e84383509a204a836"
format_stamp: "Formatted at 2026-08-10 11:30:37 on dist-test-slave-srgv"
server_key: "1ff04f0915b8373ae8755b6f4fb6ea04"
server_key_iv: "5093088c219b6c2f6185fef22ff6305c"
server_key_version: "encryptionkey@0"
I20260810 11:30:37.213627 11736 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.004s
I20260810 11:30:37.217713 11751 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:30:37.219334 11736 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 11:30:37.219519 11736 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786361401922362-8267-0/minicluster-data/ts-2/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786361401922362-8267-0/minicluster-data/ts-2/wal
uuid: "de988b5a93624b1e84383509a204a836"
format_stamp: "Formatted at 2026-08-10 11:30:37 on dist-test-slave-srgv"
server_key: "1ff04f0915b8373ae8755b6f4fb6ea04"
server_key_iv: "5093088c219b6c2f6185fef22ff6305c"
server_key_version: "encryptionkey@0"
I20260810 11:30:37.219781 11736 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786361401922362-8267-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786361401922362-8267-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786361401922362-8267-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:30:37.248859 11736 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:30:37.253907 11736 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:30:37.254199 11736 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:30:37.255188 11736 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:30:37.256579 11736 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:30:37.256662 11736 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:30:37.256762 11736 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:30:37.256807 11736 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:30:37.292388 11736 rpc_server.cc:307] RPC server started. Bound to: 127.8.18.195:40359
I20260810 11:30:37.292452 11867 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.18.195:40359 every 8 connection(s)
I20260810 11:30:37.293890 11736 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786361401922362-8267-0/minicluster-data/ts-2/data/info.pb
I20260810 11:30:37.302367 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 11736
I20260810 11:30:37.302630 8267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786361401922362-8267-0/minicluster-data/ts-2/wal/instance
I20260810 11:30:37.303259 8267 external_mini_cluster.cc:1428] Setting key 35da65233f921d10c25f7145659cc02e
Aug 10 11:30:37 dist-test-slave-srgv krb5kdc[11350](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361437, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.195@KRBTEST.COM for kudu/127.8.18.254@KRBTEST.COM
I20260810 11:30:37.318605 11870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:37.296990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.195:55397 (local address 127.8.18.254:41853)
0810 11:30:37.297595 (+ 605us) server_negotiation.cc:206] Beginning negotiation
0810 11:30:37.297601 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:30:37.298762 (+ 1161us) server_negotiation.cc:407] Connection header received
0810 11:30:37.300828 (+ 2066us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:30:37.300834 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:30:37.300958 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:30:37.301096 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:30:37.303604 (+ 2508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:37.304854 (+ 1250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:37.307224 (+ 2370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:37.307649 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:37.307757 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:37.311811 (+ 4054us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:30:37.311817 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:30:37.311820 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:30:37.311858 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:30:37.314917 (+ 3059us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:37.315812 (+ 895us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:37.315817 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:37.315819 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:37.315962 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:37.316413 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:37.316417 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:37.316418 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:37.316716 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:30:37.316850 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0810 11:30:37.317875 (+ 1025us) server_negotiation.cc:299] Negotiation successful
0810 11:30:37.318205 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":489,"thread_start_us":314,"threads_started":1}
I20260810 11:30:37.319624 11871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:37.297641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:41853 (local address 127.8.18.195:55397)
0810 11:30:37.298546 (+ 905us) negotiation.cc:107] Waiting for socket to connect
0810 11:30:37.298594 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0810 11:30:37.300448 (+ 1854us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:30:37.301394 (+ 946us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:30:37.301460 (+ 66us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:30:37.302295 (+ 835us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:30:37.303311 (+ 1016us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:30:37.303349 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:30:37.305050 (+ 1701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:30:37.305058 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:30:37.307006 (+ 1948us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:30:37.307027 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:30:37.307881 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:30:37.307886 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:30:37.308150 (+ 264us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:37.308943 (+ 793us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:30:37.308968 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:30:37.311520 (+ 2552us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:30:37.315119 (+ 3599us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:30:37.315128 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:30:37.315144 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:30:37.315662 (+ 518us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:30:37.316068 (+ 406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:30:37.316072 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:30:37.316074 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:30:37.316297 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:30:37.316834 (+ 537us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:30:37.316841 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:30:37.317221 (+ 380us) client_negotiation.cc:769] Sending connection context
0810 11:30:37.317591 (+ 370us) client_negotiation.cc:240] Negotiation successful
0810 11:30:37.318183 (+ 592us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":641,"thread_start_us":412,"threads_started":1}
I20260810 11:30:37.321228 11868 heartbeater.cc:347] Connected to a master server at 127.8.18.254:41853
I20260810 11:30:37.321600 11868 heartbeater.cc:464] Registering TS with master...
I20260810 11:30:37.322360 11868 heartbeater.cc:511] Master 127.8.18.254:41853 requested a full tablet report, sending...
I20260810 11:30:37.323971 11396 ts_manager.cc:194] Registered new tserver with Master: de988b5a93624b1e84383509a204a836 (127.8.18.195:40359)
I20260810 11:30:37.325011 11396 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.18.195@KRBTEST.COM'} at 127.8.18.195:55397
I20260810 11:30:37.333706 8267 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 10 11:30:37 dist-test-slave-srgv krb5kdc[11350](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361437, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 10 11:30:37 dist-test-slave-srgv krb5kdc[11350](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361437, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.8.18.254@KRBTEST.COM
I20260810 11:30:37.392506 11870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:37.375876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35746 (local address 127.8.18.254:41853)
0810 11:30:37.376121 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0810 11:30:37.376130 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 11:30:37.376823 (+ 693us) server_negotiation.cc:407] Connection header received
0810 11:30:37.377106 (+ 283us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:30:37.377112 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:30:37.377199 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:30:37.377322 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:30:37.379026 (+ 1704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:37.379838 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:37.381232 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:37.381737 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:37.381889 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:37.386607 (+ 4718us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:30:37.386632 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:30:37.386635 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:30:37.386693 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:30:37.389525 (+ 2832us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:37.390348 (+ 823us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:37.390353 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:37.390355 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:37.390509 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:37.391023 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:37.391028 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:37.391030 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:37.391409 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:30:37.391661 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0810 11:30:37.392023 (+ 362us) server_negotiation.cc:299] Negotiation successful
0810 11:30:37.392262 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
W20260810 11:30:37.393175 11396 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:35746
I20260810 11:30:37.395849 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 11458
Aug 10 11:30:37 dist-test-slave-srgv krb5kdc[11350](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361437, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.8.18.193@KRBTEST.COM
I20260810 11:30:37.415057 11880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:37.396978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47416 (local address 127.8.18.193:42819)
0810 11:30:37.397641 (+ 663us) server_negotiation.cc:206] Beginning negotiation
0810 11:30:37.397649 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:30:37.397676 (+ 27us) server_negotiation.cc:407] Connection header received
0810 11:30:37.397739 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:30:37.397746 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:30:37.397974 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:30:37.398187 (+ 213us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:30:37.399611 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:37.401220 (+ 1609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:37.402649 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:37.403304 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:37.403432 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:37.408396 (+ 4964us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:30:37.408415 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:30:37.408435 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:30:37.408508 (+ 73us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:30:37.411787 (+ 3279us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:37.412690 (+ 903us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:37.412699 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:37.412701 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:37.412864 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:37.413318 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:37.413322 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:37.413325 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:37.413872 (+ 547us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:30:37.414130 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0810 11:30:37.414485 (+ 355us) server_negotiation.cc:299] Negotiation successful
0810 11:30:37.414745 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":495,"thread_start_us":325,"threads_started":1}
W20260810 11:30:37.416481 11586 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:47416
W20260810 11:30:37.417910 8267 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 11458: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 11:30:37.418157 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 11458
I20260810 11:30:37.435756 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 11598
Aug 10 11:30:37 dist-test-slave-srgv krb5kdc[11350](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361437, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.8.18.194@KRBTEST.COM
I20260810 11:30:37.455235 11881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:37.436984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44100 (local address 127.8.18.194:35929)
0810 11:30:37.437690 (+ 706us) server_negotiation.cc:206] Beginning negotiation
0810 11:30:37.437702 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 11:30:37.437742 (+ 40us) server_negotiation.cc:407] Connection header received
0810 11:30:37.437809 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:30:37.437815 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:30:37.438039 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:30:37.438185 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:30:37.439665 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:37.440827 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:37.442452 (+ 1625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:37.443046 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:37.443170 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:37.448147 (+ 4977us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:30:37.448165 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:30:37.448186 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:30:37.448254 (+ 68us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:30:37.451716 (+ 3462us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:37.452679 (+ 963us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:37.452689 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:37.452691 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:37.452922 (+ 231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:37.453531 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:37.453538 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:37.453542 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:37.453977 (+ 435us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:30:37.454249 (+ 272us) server_negotiation.cc:1035] Waiting for connection context
0810 11:30:37.454557 (+ 308us) server_negotiation.cc:299] Negotiation successful
0810 11:30:37.454789 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":503,"thread_start_us":340,"threads_started":1}
W20260810 11:30:37.456609 11725 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:44100
W20260810 11:30:37.457909 8267 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 11598: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 11:30:37.458145 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 11598
I20260810 11:30:37.475255 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 11736
Aug 10 11:30:37 dist-test-slave-srgv krb5kdc[11350](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361437, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.8.18.195@KRBTEST.COM
I20260810 11:30:37.495466 11882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:37.476343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45130 (local address 127.8.18.195:40359)
0810 11:30:37.477006 (+ 663us) server_negotiation.cc:206] Beginning negotiation
0810 11:30:37.477014 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:30:37.477045 (+ 31us) server_negotiation.cc:407] Connection header received
0810 11:30:37.477105 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:30:37.477111 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:30:37.477328 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:30:37.477536 (+ 208us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:30:37.478945 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:37.480036 (+ 1091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:37.481375 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:37.482093 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:37.482222 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:37.487985 (+ 5763us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:30:37.488006 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:30:37.488028 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:30:37.488081 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:30:37.492165 (+ 4084us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:37.493099 (+ 934us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:37.493110 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:37.493114 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:37.493293 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:37.493860 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:37.493866 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:37.493869 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:37.494338 (+ 469us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:30:37.494643 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0810 11:30:37.494941 (+ 298us) server_negotiation.cc:299] Negotiation successful
0810 11:30:37.495171 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":517,"thread_start_us":361,"threads_started":1}
W20260810 11:30:37.496814 11863 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:45130
W20260810 11:30:37.498030 8267 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 11736: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 11:30:37.498219 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 11736
I20260810 11:30:37.515314 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 11363
I20260810 11:30:37.516773 11426 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:30:37.623906 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 11363
2026-08-10T11:30:37Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (3996 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786361401922362-8267-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 11:30:37 dist-test-slave-srgv krb5kdc[11886](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 11:30:37 dist-test-slave-srgv krb5kdc[11886](info): set up 2 sockets
Aug 10 11:30:37 dist-test-slave-srgv krb5kdc[11886](info): commencing operation
krb5kdc: starting...
W20260810 11:30:38.813031 8267 mini_kdc.cc:121] Time spent starting KDC: real 1.144s user 0.003s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786361401922362-8267-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786361401922362-8267-0/krb5kdc/test-user.keytab.
Aug 10 11:30:38 dist-test-slave-srgv krb5kdc[11886](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361438, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T11:30:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T11:30:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.18.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.18.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.18.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.254.keytab.
Entry for principal kudu/127.8.18.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.254.keytab.
I20260810 11:30:39.053462 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786361401922362-8267-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786361401922362-8267-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786361401922362-8267-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.18.254:39125
--webserver_interface=127.8.18.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.254.keytab
--principal=kudu/127.8.18.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.18.212:44791
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.18.254:39125
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786361401922362-8267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786361401922362-8267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786361401922362-8267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:30:39.399672 11899 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:30:39.400108 11899 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:30:39.400288 11899 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:30:39.408003 11899 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 11:30:39.408202 11899 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:30:39.408326 11899 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:30:39.408432 11899 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 11:30:39.408521 11899 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 11:30:39.419090 11899 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.18.212:44791
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786361401922362-8267-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786361401922362-8267-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.18.254:39125
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.254.keytab
--principal=kudu/127.8.18.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.18.254:39125
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.18.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786361401922362-8267-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:30:39.420588 11899 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:30:39.422089 11899 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:30:39.435735 11905 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:30:39.435679 11904 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:30:39.436321 11907 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:30:39.436504 11899 server_base.cc:1034] running on GCE node
I20260810 11:30:39.437569 11899 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:30:39.439288 11899 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:30:39.440569 11899 hybrid_clock.cc:648] HybridClock initialized: now 1786361439440515 us; error 76 us; skew 500 ppm
I20260810 11:30:39.440984 11899 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:30:39 dist-test-slave-srgv krb5kdc[11886](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361439, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:30:39.444641 11899 init.cc:377] Logged in from keytab as kudu/127.8.18.254@KRBTEST.COM (short username kudu)
I20260810 11:30:39.446630 11899 webserver.cc:460] Webserver started at http://127.8.18.254:45183/ using document root <none> and password file <none>
I20260810 11:30:39.447408 11899 fs_manager.cc:359] Metadata directory not provided
I20260810 11:30:39.447535 11899 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:30:39.447839 11899 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:30:39.451268 11899 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786361401922362-8267-0/minicluster-data/master-0/data/instance:
uuid: "1828256289e348cca12655b2b3c27abe"
format_stamp: "Formatted at 2026-08-10 11:30:39 on dist-test-slave-srgv"
server_key: "1bd418baf4987c69d2028923f6f03818"
server_key_iv: "10ece43996ab0b3b636f45a74bbf9fc0"
server_key_version: "encryptionkey@0"
I20260810 11:30:39.452099 11899 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786361401922362-8267-0/minicluster-data/master-0/wal/instance:
uuid: "1828256289e348cca12655b2b3c27abe"
format_stamp: "Formatted at 2026-08-10 11:30:39 on dist-test-slave-srgv"
server_key: "1bd418baf4987c69d2028923f6f03818"
server_key_iv: "10ece43996ab0b3b636f45a74bbf9fc0"
server_key_version: "encryptionkey@0"
I20260810 11:30:39.458195 11899 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260810 11:30:39.462709 11916 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:30:39.464324 11899 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.005s sys 0.000s
I20260810 11:30:39.464537 11899 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786361401922362-8267-0/minicluster-data/master-0/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786361401922362-8267-0/minicluster-data/master-0/wal
uuid: "1828256289e348cca12655b2b3c27abe"
format_stamp: "Formatted at 2026-08-10 11:30:39 on dist-test-slave-srgv"
server_key: "1bd418baf4987c69d2028923f6f03818"
server_key_iv: "10ece43996ab0b3b636f45a74bbf9fc0"
server_key_version: "encryptionkey@0"
I20260810 11:30:39.464775 11899 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786361401922362-8267-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786361401922362-8267-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786361401922362-8267-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:30:39.486418 11899 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:30:39.491163 11899 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:30:39.491510 11899 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:30:39.512722 11899 rpc_server.cc:307] RPC server started. Bound to: 127.8.18.254:39125
I20260810 11:30:39.512755 11970 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.18.254:39125 every 8 connection(s)
I20260810 11:30:39.514479 11899 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
I20260810 11:30:39.518149 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 11899
I20260810 11:30:39.518417 8267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786361401922362-8267-0/minicluster-data/master-0/wal/instance
I20260810 11:30:39.518915 11971 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:30:39.519140 8267 external_mini_cluster.cc:1428] Setting key 31fe3290deb25643f828a309dcda1232
I20260810 11:30:39.535125 11971 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1828256289e348cca12655b2b3c27abe: Bootstrap starting.
Aug 10 11:30:39 dist-test-slave-srgv krb5kdc[11886](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361438, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.18.254@KRBTEST.COM
I20260810 11:30:39.539641 11971 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1828256289e348cca12655b2b3c27abe: Neither blocks nor log segments found. Creating new log.
I20260810 11:30:39.541172 11971 log.cc:824] T 00000000000000000000000000000000 P 1828256289e348cca12655b2b3c27abe: Log is configured to *not* fsync() on all Append() calls
I20260810 11:30:39.545549 11971 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1828256289e348cca12655b2b3c27abe: No bootstrap required, opened a new log
I20260810 11:30:39.546993 11974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:39.521341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33378 (local address 127.8.18.254:39125)
0810 11:30:39.522310 (+ 969us) server_negotiation.cc:206] Beginning negotiation
0810 11:30:39.522327 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0810 11:30:39.522387 (+ 60us) server_negotiation.cc:407] Connection header received
0810 11:30:39.524614 (+ 2227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:30:39.524649 (+ 35us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:30:39.525271 (+ 622us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:30:39.525821 (+ 550us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:30:39.527388 (+ 1567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:39.529466 (+ 2078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:39.530869 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:39.532140 (+ 1271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:39.532371 (+ 231us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:39.536920 (+ 4549us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:30:39.536936 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:30:39.536967 (+ 31us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:30:39.537031 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:30:39.541433 (+ 4402us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:39.542567 (+ 1134us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:39.542576 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:39.542585 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:39.542808 (+ 223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:39.543317 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:39.543323 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:39.543326 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:39.544157 (+ 831us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:30:39.544509 (+ 352us) server_negotiation.cc:1035] Waiting for connection context
0810 11:30:39.544895 (+ 386us) server_negotiation.cc:299] Negotiation successful
0810 11:30:39.545454 (+ 559us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":694,"thread_start_us":393,"threads_started":1}
I20260810 11:30:39.554562 11971 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1828256289e348cca12655b2b3c27abe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1828256289e348cca12655b2b3c27abe" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 39125 } }
I20260810 11:30:39.554961 11971 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1828256289e348cca12655b2b3c27abe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:30:39.555037 11971 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1828256289e348cca12655b2b3c27abe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1828256289e348cca12655b2b3c27abe, State: Initialized, Role: FOLLOWER
I20260810 11:30:39.555559 11971 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1828256289e348cca12655b2b3c27abe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1828256289e348cca12655b2b3c27abe" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 39125 } }
I20260810 11:30:39.555872 11971 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1828256289e348cca12655b2b3c27abe [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 11:30:39.556041 11971 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1828256289e348cca12655b2b3c27abe [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 11:30:39.556248 11971 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1828256289e348cca12655b2b3c27abe [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:30:39.558996 11971 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1828256289e348cca12655b2b3c27abe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1828256289e348cca12655b2b3c27abe" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 39125 } }
I20260810 11:30:39.559521 11971 leader_election.cc:302] T 00000000000000000000000000000000 P 1828256289e348cca12655b2b3c27abe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1828256289e348cca12655b2b3c27abe; no voters:
I20260810 11:30:39.560102 11971 leader_election.cc:288] T 00000000000000000000000000000000 P 1828256289e348cca12655b2b3c27abe [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 11:30:39.560408 11976 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1828256289e348cca12655b2b3c27abe [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:30:39.561196 11976 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1828256289e348cca12655b2b3c27abe [term 1 LEADER]: Becoming Leader. State: Replica: 1828256289e348cca12655b2b3c27abe, State: Running, Role: LEADER
I20260810 11:30:39.561880 11976 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1828256289e348cca12655b2b3c27abe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1828256289e348cca12655b2b3c27abe" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 39125 } }
I20260810 11:30:39.562744 11971 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1828256289e348cca12655b2b3c27abe [sys.catalog]: configured and running, proceeding with master startup.
I20260810 11:30:39.565027 11977 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1828256289e348cca12655b2b3c27abe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1828256289e348cca12655b2b3c27abe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1828256289e348cca12655b2b3c27abe" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 39125 } } }
I20260810 11:30:39.565009 11978 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1828256289e348cca12655b2b3c27abe [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1828256289e348cca12655b2b3c27abe. Latest consensus state: current_term: 1 leader_uuid: "1828256289e348cca12655b2b3c27abe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1828256289e348cca12655b2b3c27abe" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 39125 } } }
I20260810 11:30:39.565570 11977 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1828256289e348cca12655b2b3c27abe [sys.catalog]: This master's current role is: LEADER
I20260810 11:30:39.565569 11978 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1828256289e348cca12655b2b3c27abe [sys.catalog]: This master's current role is: LEADER
I20260810 11:30:39.568673 11982 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 11:30:39.574465 11982 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 11:30:39.585553 11982 catalog_manager.cc:1347] Generated new cluster ID: e14c6853076b46b0a09432f25d48ce4b
I20260810 11:30:39.585724 11982 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 11:30:39.597126 11982 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 11:30:39.598636 11982 catalog_manager.cc:1504] Loading token signing keys...
I20260810 11:30:39.611562 11982 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1828256289e348cca12655b2b3c27abe: Generated new TSK 0
I20260810 11:30:39.612521 11982 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.18.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.18.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.18.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.193.keytab.
Entry for principal kudu/127.8.18.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.193.keytab.
I20260810 11:30:39.674760 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786361401922362-8267-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786361401922362-8267-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786361401922362-8267-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786361401922362-8267-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.18.193:0
--local_ip_for_outbound_sockets=127.8.18.193
--webserver_interface=127.8.18.193
--webserver_port=0
--tserver_master_addrs=127.8.18.254:39125
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.193.keytab
--principal=kudu/127.8.18.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.18.212:44791
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786361401922362-8267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786361401922362-8267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786361401922362-8267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:30:40.041709 11997 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:30:40.042023 11997 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:30:40.042127 11997 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:30:40.049966 11997 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:30:40.050091 11997 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:30:40.050209 11997 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.18.193
I20260810 11:30:40.060601 11997 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.18.212:44791
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786361401922362-8267-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786361401922362-8267-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.193.keytab
--principal=kudu/127.8.18.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.18.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786361401922362-8267-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.18.193
--webserver_port=0
--tserver_master_addrs=127.8.18.254:39125
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.18.193
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786361401922362-8267-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:30:40.062058 11997 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:30:40.063746 11997 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:30:40.078572 12002 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:30:40.078785 12003 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:30:40.078608 12005 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:30:40.079623 11997 server_base.cc:1034] running on GCE node
I20260810 11:30:40.080369 11997 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:30:40.081645 11997 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:30:40.082901 11997 hybrid_clock.cc:648] HybridClock initialized: now 1786361440082845 us; error 49 us; skew 500 ppm
I20260810 11:30:40.083292 11997 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:30:40 dist-test-slave-srgv krb5kdc[11886](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361440, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:30:40.087177 11997 init.cc:377] Logged in from keytab as kudu/127.8.18.193@KRBTEST.COM (short username kudu)
I20260810 11:30:40.088886 11997 webserver.cc:460] Webserver started at http://127.8.18.193:45511/ using document root <none> and password file <none>
I20260810 11:30:40.089723 11997 fs_manager.cc:359] Metadata directory not provided
I20260810 11:30:40.089849 11997 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:30:40.090145 11997 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:30:40.093508 11997 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786361401922362-8267-0/minicluster-data/ts-0/data/instance:
uuid: "b572e931f5524a54b8f296807c02e795"
format_stamp: "Formatted at 2026-08-10 11:30:40 on dist-test-slave-srgv"
server_key: "5de8155a8f567aa2e3b9a11df423de3e"
server_key_iv: "5cba409b9757905d0da3b19fdd30bf81"
server_key_version: "encryptionkey@0"
I20260810 11:30:40.094451 11997 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786361401922362-8267-0/minicluster-data/ts-0/wal/instance:
uuid: "b572e931f5524a54b8f296807c02e795"
format_stamp: "Formatted at 2026-08-10 11:30:40 on dist-test-slave-srgv"
server_key: "5de8155a8f567aa2e3b9a11df423de3e"
server_key_iv: "5cba409b9757905d0da3b19fdd30bf81"
server_key_version: "encryptionkey@0"
I20260810 11:30:40.100209 11997 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260810 11:30:40.104310 12012 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:30:40.105906 11997 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 11:30:40.106113 11997 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786361401922362-8267-0/minicluster-data/ts-0/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786361401922362-8267-0/minicluster-data/ts-0/wal
uuid: "b572e931f5524a54b8f296807c02e795"
format_stamp: "Formatted at 2026-08-10 11:30:40 on dist-test-slave-srgv"
server_key: "5de8155a8f567aa2e3b9a11df423de3e"
server_key_iv: "5cba409b9757905d0da3b19fdd30bf81"
server_key_version: "encryptionkey@0"
I20260810 11:30:40.106357 11997 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786361401922362-8267-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786361401922362-8267-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786361401922362-8267-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:30:40.123965 11997 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:30:40.129020 11997 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:30:40.129369 11997 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:30:40.130518 11997 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:30:40.132113 11997 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:30:40.132221 11997 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:30:40.132395 11997 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:30:40.132468 11997 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:30:40.168268 11997 rpc_server.cc:307] RPC server started. Bound to: 127.8.18.193:40525
I20260810 11:30:40.168298 12129 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.18.193:40525 every 8 connection(s)
I20260810 11:30:40.169960 11997 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786361401922362-8267-0/minicluster-data/ts-0/data/info.pb
I20260810 11:30:40.172832 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 11997
I20260810 11:30:40.173072 8267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786361401922362-8267-0/minicluster-data/ts-0/wal/instance
I20260810 11:30:40.173974 8267 external_mini_cluster.cc:1428] Setting key 77c23f70a57c5088c9938b37de09f414
Aug 10 11:30:40 dist-test-slave-srgv krb5kdc[11886](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361440, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.193@KRBTEST.COM for kudu/127.8.18.254@KRBTEST.COM
I20260810 11:30:40.194478 12132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:40.172880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.193:43031 (local address 127.8.18.254:39125)
0810 11:30:40.173816 (+ 936us) server_negotiation.cc:206] Beginning negotiation
0810 11:30:40.173821 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:30:40.174954 (+ 1133us) server_negotiation.cc:407] Connection header received
0810 11:30:40.177108 (+ 2154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:30:40.177113 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:30:40.177209 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:30:40.177344 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:30:40.179035 (+ 1691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:40.179787 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:40.181476 (+ 1689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:40.181900 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:40.182049 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:40.186548 (+ 4499us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:30:40.186558 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:30:40.186565 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:30:40.186615 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:30:40.190488 (+ 3873us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:40.191456 (+ 968us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:40.191466 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:40.191472 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:40.191643 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:40.192161 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:40.192169 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:40.192175 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:40.192601 (+ 426us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:30:40.192782 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0810 11:30:40.193635 (+ 853us) server_negotiation.cc:299] Negotiation successful
0810 11:30:40.193939 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":822,"thread_start_us":702,"threads_started":1}
I20260810 11:30:40.195119 12133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:40.173770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:39125 (local address 127.8.18.193:43031)
0810 11:30:40.174724 (+ 954us) negotiation.cc:107] Waiting for socket to connect
0810 11:30:40.174784 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0810 11:30:40.176802 (+ 2018us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:30:40.177642 (+ 840us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:30:40.177651 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:30:40.178215 (+ 564us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:30:40.178836 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:30:40.178854 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:30:40.179967 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:30:40.179971 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:30:40.181283 (+ 1312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:30:40.181296 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:30:40.182184 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:30:40.182192 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:30:40.182446 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:40.183263 (+ 817us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:30:40.183288 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:30:40.186251 (+ 2963us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:30:40.190738 (+ 4487us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:30:40.190746 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:30:40.190763 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:30:40.191278 (+ 515us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:30:40.191798 (+ 520us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:30:40.191802 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:30:40.191804 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:30:40.192014 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:30:40.192760 (+ 746us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:30:40.192767 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:30:40.193139 (+ 372us) client_negotiation.cc:769] Sending connection context
0810 11:30:40.193389 (+ 250us) client_negotiation.cc:240] Negotiation successful
0810 11:30:40.193902 (+ 513us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":627,"thread_start_us":384,"threads_started":1}
I20260810 11:30:40.196928 12130 heartbeater.cc:347] Connected to a master server at 127.8.18.254:39125
WARNING: no policy specified for kudu/127.8.18.194@KRBTEST.COM; defaulting to no policy
I20260810 11:30:40.197261 12130 heartbeater.cc:464] Registering TS with master...
I20260810 11:30:40.198174 12130 heartbeater.cc:511] Master 127.8.18.254:39125 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.18.194@KRBTEST.COM" created.
I20260810 11:30:40.200807 11933 ts_manager.cc:194] Registered new tserver with Master: b572e931f5524a54b8f296807c02e795 (127.8.18.193:40525)
I20260810 11:30:40.203233 11933 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.18.193@KRBTEST.COM'} at 127.8.18.193:43031
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.18.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.194.keytab.
Entry for principal kudu/127.8.18.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.194.keytab.
I20260810 11:30:40.231606 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786361401922362-8267-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786361401922362-8267-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786361401922362-8267-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786361401922362-8267-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.18.194:0
--local_ip_for_outbound_sockets=127.8.18.194
--webserver_interface=127.8.18.194
--webserver_port=0
--tserver_master_addrs=127.8.18.254:39125
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.194.keytab
--principal=kudu/127.8.18.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.18.212:44791
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786361401922362-8267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786361401922362-8267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786361401922362-8267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:30:40.586153 12136 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:30:40.586474 12136 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:30:40.586577 12136 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:30:40.594022 12136 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:30:40.594161 12136 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:30:40.594287 12136 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.18.194
I20260810 11:30:40.604864 12136 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.18.212:44791
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786361401922362-8267-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786361401922362-8267-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.194.keytab
--principal=kudu/127.8.18.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.18.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786361401922362-8267-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.18.194
--webserver_port=0
--tserver_master_addrs=127.8.18.254:39125
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.18.194
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786361401922362-8267-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:30:40.606259 12136 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:30:40.607653 12136 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:30:40.622316 12142 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:30:40.622268 12141 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:30:40.622298 12144 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:30:40.623432 12136 server_base.cc:1034] running on GCE node
I20260810 11:30:40.624169 12136 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:30:40.625160 12136 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:30:40.626389 12136 hybrid_clock.cc:648] HybridClock initialized: now 1786361440626329 us; error 47 us; skew 500 ppm
I20260810 11:30:40.626757 12136 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:30:40 dist-test-slave-srgv krb5kdc[11886](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361440, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:30:40.635622 12136 init.cc:377] Logged in from keytab as kudu/127.8.18.194@KRBTEST.COM (short username kudu)
I20260810 11:30:40.637327 12136 webserver.cc:460] Webserver started at http://127.8.18.194:41913/ using document root <none> and password file <none>
I20260810 11:30:40.638150 12136 fs_manager.cc:359] Metadata directory not provided
I20260810 11:30:40.638271 12136 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:30:40.638557 12136 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:30:40.641851 12136 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786361401922362-8267-0/minicluster-data/ts-1/data/instance:
uuid: "98d4e85679b54d948b6390dea368bbb7"
format_stamp: "Formatted at 2026-08-10 11:30:40 on dist-test-slave-srgv"
server_key: "567ab2391cc7ea1763443d6ec0437bc5"
server_key_iv: "dc352991d698569698a8521bead821d3"
server_key_version: "encryptionkey@0"
I20260810 11:30:40.642721 12136 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786361401922362-8267-0/minicluster-data/ts-1/wal/instance:
uuid: "98d4e85679b54d948b6390dea368bbb7"
format_stamp: "Formatted at 2026-08-10 11:30:40 on dist-test-slave-srgv"
server_key: "567ab2391cc7ea1763443d6ec0437bc5"
server_key_iv: "dc352991d698569698a8521bead821d3"
server_key_version: "encryptionkey@0"
I20260810 11:30:40.648937 12136 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260810 11:30:40.653282 12151 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:30:40.655158 12136 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260810 11:30:40.655395 12136 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786361401922362-8267-0/minicluster-data/ts-1/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786361401922362-8267-0/minicluster-data/ts-1/wal
uuid: "98d4e85679b54d948b6390dea368bbb7"
format_stamp: "Formatted at 2026-08-10 11:30:40 on dist-test-slave-srgv"
server_key: "567ab2391cc7ea1763443d6ec0437bc5"
server_key_iv: "dc352991d698569698a8521bead821d3"
server_key_version: "encryptionkey@0"
I20260810 11:30:40.655644 12136 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786361401922362-8267-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786361401922362-8267-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786361401922362-8267-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:30:40.673929 12136 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:30:40.679076 12136 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:30:40.679440 12136 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:30:40.680562 12136 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:30:40.682217 12136 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:30:40.682305 12136 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:30:40.682479 12136 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:30:40.682555 12136 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:30:40.717481 12136 rpc_server.cc:307] RPC server started. Bound to: 127.8.18.194:44685
I20260810 11:30:40.717540 12266 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.18.194:44685 every 8 connection(s)
I20260810 11:30:40.718961 12136 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786361401922362-8267-0/minicluster-data/ts-1/data/info.pb
I20260810 11:30:40.726361 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 12136
I20260810 11:30:40.726539 8267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786361401922362-8267-0/minicluster-data/ts-1/wal/instance
I20260810 11:30:40.727016 8267 external_mini_cluster.cc:1428] Setting key 7c50981336edc03d496e1744ea6951ef
Aug 10 11:30:40 dist-test-slave-srgv krb5kdc[11886](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361440, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.194@KRBTEST.COM for kudu/127.8.18.254@KRBTEST.COM
I20260810 11:30:40.741972 12269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:40.721681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.194:37207 (local address 127.8.18.254:39125)
0810 11:30:40.722144 (+ 463us) server_negotiation.cc:206] Beginning negotiation
0810 11:30:40.722150 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:30:40.723178 (+ 1028us) server_negotiation.cc:407] Connection header received
0810 11:30:40.725099 (+ 1921us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:30:40.725104 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:30:40.725178 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:30:40.725291 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:30:40.727236 (+ 1945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:40.728114 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:40.729991 (+ 1877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:40.730433 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:40.730594 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:40.734913 (+ 4319us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:30:40.734923 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:30:40.734931 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:30:40.734985 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:30:40.738025 (+ 3040us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:40.739164 (+ 1139us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:40.739168 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:40.739170 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:40.739312 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:40.739932 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:40.739936 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:40.739938 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:40.740245 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:30:40.740371 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0810 11:30:40.741401 (+ 1030us) server_negotiation.cc:299] Negotiation successful
0810 11:30:40.741683 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":334,"thread_start_us":215,"threads_started":1}
I20260810 11:30:40.743752 12270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:40.722275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:39125 (local address 127.8.18.194:37207)
0810 11:30:40.723001 (+ 726us) negotiation.cc:107] Waiting for socket to connect
0810 11:30:40.723040 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0810 11:30:40.724803 (+ 1763us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:30:40.725598 (+ 795us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:30:40.725606 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:30:40.726228 (+ 622us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:30:40.726973 (+ 745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:30:40.726992 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:30:40.728311 (+ 1319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:30:40.728315 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:30:40.729831 (+ 1516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:30:40.729846 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:30:40.730685 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:30:40.730692 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:30:40.730945 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:40.731741 (+ 796us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:30:40.731767 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:30:40.734553 (+ 2786us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:30:40.738217 (+ 3664us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:30:40.738232 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:30:40.738256 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:30:40.739007 (+ 751us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:30:40.739441 (+ 434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:30:40.739448 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:30:40.739455 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:30:40.739792 (+ 337us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:30:40.740392 (+ 600us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:30:40.740405 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:30:40.740953 (+ 548us) client_negotiation.cc:769] Sending connection context
0810 11:30:40.741299 (+ 346us) client_negotiation.cc:240] Negotiation successful
0810 11:30:40.742038 (+ 739us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":482,"thread_start_us":298,"threads_started":1}
I20260810 11:30:40.745764 12267 heartbeater.cc:347] Connected to a master server at 127.8.18.254:39125
I20260810 11:30:40.746165 12267 heartbeater.cc:464] Registering TS with master...
I20260810 11:30:40.747263 12267 heartbeater.cc:511] Master 127.8.18.254:39125 requested a full tablet report, sending...
I20260810 11:30:40.749292 11933 ts_manager.cc:194] Registered new tserver with Master: 98d4e85679b54d948b6390dea368bbb7 (127.8.18.194:44685)
I20260810 11:30:40.750195 11933 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.18.194@KRBTEST.COM'} at 127.8.18.194:37207
WARNING: no policy specified for kudu/127.8.18.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.18.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.18.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.195.keytab.
Entry for principal kudu/127.8.18.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.195.keytab.
I20260810 11:30:40.784811 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786361401922362-8267-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786361401922362-8267-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786361401922362-8267-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786361401922362-8267-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.18.195:0
--local_ip_for_outbound_sockets=127.8.18.195
--webserver_interface=127.8.18.195
--webserver_port=0
--tserver_master_addrs=127.8.18.254:39125
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.195.keytab
--principal=kudu/127.8.18.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.18.212:44791
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786361401922362-8267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786361401922362-8267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786361401922362-8267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:30:41.137780 12273 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:30:41.138173 12273 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:30:41.138363 12273 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:30:41.146178 12273 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:30:41.146360 12273 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:30:41.146555 12273 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.18.195
I20260810 11:30:41.156991 12273 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.18.212:44791
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786361401922362-8267-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786361401922362-8267-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.195.keytab
--principal=kudu/127.8.18.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.18.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786361401922362-8267-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.18.195
--webserver_port=0
--tserver_master_addrs=127.8.18.254:39125
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.18.195
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786361401922362-8267-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:30:41.158476 12273 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:30:41.159937 12273 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:30:41.179242 12279 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:30:41.180763 12278 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:30:41.181221 12273 server_base.cc:1034] running on GCE node
W20260810 11:30:41.181876 12281 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:30:41.182674 12273 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:30:41.185111 12273 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:30:41.186355 12273 hybrid_clock.cc:648] HybridClock initialized: now 1786361441186296 us; error 63 us; skew 500 ppm
I20260810 11:30:41.186690 12273 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:30:41 dist-test-slave-srgv krb5kdc[11886](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361441, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:30:41.190497 12273 init.cc:377] Logged in from keytab as kudu/127.8.18.195@KRBTEST.COM (short username kudu)
I20260810 11:30:41.192284 12273 webserver.cc:460] Webserver started at http://127.8.18.195:34965/ using document root <none> and password file <none>
I20260810 11:30:41.193055 12273 fs_manager.cc:359] Metadata directory not provided
I20260810 11:30:41.193174 12273 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:30:41.193564 12273 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:30:41.196794 12273 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786361401922362-8267-0/minicluster-data/ts-2/data/instance:
uuid: "387fc08366df4629be4c0d1be86facbb"
format_stamp: "Formatted at 2026-08-10 11:30:41 on dist-test-slave-srgv"
server_key: "5945c8ad338342b88612fdabfaf91190"
server_key_iv: "35cf3786198af6419a601ee0f8fecd8f"
server_key_version: "encryptionkey@0"
I20260810 11:30:41.197733 12273 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786361401922362-8267-0/minicluster-data/ts-2/wal/instance:
uuid: "387fc08366df4629be4c0d1be86facbb"
format_stamp: "Formatted at 2026-08-10 11:30:41 on dist-test-slave-srgv"
server_key: "5945c8ad338342b88612fdabfaf91190"
server_key_iv: "35cf3786198af6419a601ee0f8fecd8f"
server_key_version: "encryptionkey@0"
I20260810 11:30:41.203933 12273 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.001s sys 0.004s
I20260810 11:30:41.207309 12130 heartbeater.cc:503] Master 127.8.18.254:39125 was elected leader, sending a full tablet report...
I20260810 11:30:41.208833 12288 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:30:41.210801 12273 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260810 11:30:41.210976 12273 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786361401922362-8267-0/minicluster-data/ts-2/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786361401922362-8267-0/minicluster-data/ts-2/wal
uuid: "387fc08366df4629be4c0d1be86facbb"
format_stamp: "Formatted at 2026-08-10 11:30:41 on dist-test-slave-srgv"
server_key: "5945c8ad338342b88612fdabfaf91190"
server_key_iv: "35cf3786198af6419a601ee0f8fecd8f"
server_key_version: "encryptionkey@0"
I20260810 11:30:41.211184 12273 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786361401922362-8267-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786361401922362-8267-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786361401922362-8267-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:30:41.231999 12273 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:30:41.236922 12273 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:30:41.237197 12273 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:30:41.238348 12273 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:30:41.239951 12273 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:30:41.240036 12273 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:30:41.240131 12273 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:30:41.240176 12273 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:30:41.276005 12273 rpc_server.cc:307] RPC server started. Bound to: 127.8.18.195:38707
I20260810 11:30:41.276039 12403 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.18.195:38707 every 8 connection(s)
I20260810 11:30:41.277702 12273 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786361401922362-8267-0/minicluster-data/ts-2/data/info.pb
I20260810 11:30:41.279870 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 12273
I20260810 11:30:41.280066 8267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786361401922362-8267-0/minicluster-data/ts-2/wal/instance
I20260810 11:30:41.280601 8267 external_mini_cluster.cc:1428] Setting key 736fe28719a96892ac38d781d0d33bba
Aug 10 11:30:41 dist-test-slave-srgv krb5kdc[11886](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361441, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.195@KRBTEST.COM for kudu/127.8.18.254@KRBTEST.COM
I20260810 11:30:41.302219 12406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:41.281893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.195:44285 (local address 127.8.18.254:39125)
0810 11:30:41.282580 (+ 687us) server_negotiation.cc:206] Beginning negotiation
0810 11:30:41.282588 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:30:41.283452 (+ 864us) server_negotiation.cc:407] Connection header received
0810 11:30:41.285708 (+ 2256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:30:41.285713 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:30:41.285789 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:30:41.285909 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:30:41.287699 (+ 1790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:41.288591 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:41.290412 (+ 1821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:41.290891 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:41.291020 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:41.294964 (+ 3944us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:30:41.294971 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:30:41.294975 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:30:41.295018 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:30:41.298764 (+ 3746us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:41.299700 (+ 936us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:41.299705 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:41.299708 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:41.299868 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:41.300373 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:41.300377 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:41.300379 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:41.300701 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:30:41.300842 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0810 11:30:41.301671 (+ 829us) server_negotiation.cc:299] Negotiation successful
0810 11:30:41.301873 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":552,"thread_start_us":397,"threads_started":1}
I20260810 11:30:41.303273 12407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:41.281696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:39125 (local address 127.8.18.195:44285)
0810 11:30:41.283244 (+ 1548us) negotiation.cc:107] Waiting for socket to connect
0810 11:30:41.283301 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0810 11:30:41.285332 (+ 2031us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:30:41.286149 (+ 817us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:30:41.286159 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:30:41.286790 (+ 631us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:30:41.287444 (+ 654us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:30:41.287465 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:30:41.288756 (+ 1291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:30:41.288760 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:30:41.290184 (+ 1424us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:30:41.290199 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:30:41.291124 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:30:41.291129 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:30:41.291381 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:41.292191 (+ 810us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:30:41.292222 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:30:41.294656 (+ 2434us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:30:41.298960 (+ 4304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:30:41.298969 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:30:41.298985 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:30:41.299532 (+ 547us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:30:41.299979 (+ 447us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:30:41.299982 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:30:41.299984 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:30:41.300187 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:30:41.300820 (+ 633us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:30:41.300827 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:30:41.301185 (+ 358us) client_negotiation.cc:769] Sending connection context
0810 11:30:41.301547 (+ 362us) client_negotiation.cc:240] Negotiation successful
0810 11:30:41.301986 (+ 439us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1181,"thread_start_us":882,"threads_started":1}
I20260810 11:30:41.305167 12404 heartbeater.cc:347] Connected to a master server at 127.8.18.254:39125
I20260810 11:30:41.305567 12404 heartbeater.cc:464] Registering TS with master...
I20260810 11:30:41.306378 12404 heartbeater.cc:511] Master 127.8.18.254:39125 requested a full tablet report, sending...
I20260810 11:30:41.308306 11933 ts_manager.cc:194] Registered new tserver with Master: 387fc08366df4629be4c0d1be86facbb (127.8.18.195:38707)
I20260810 11:30:41.309532 11933 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.18.195@KRBTEST.COM'} at 127.8.18.195:44285
I20260810 11:30:41.310521 8267 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 10 11:30:41 dist-test-slave-srgv krb5kdc[11886](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361441, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 10 11:30:41 dist-test-slave-srgv krb5kdc[11886](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361441, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.18.193@KRBTEST.COM
I20260810 11:30:41.369969 12412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:41.351563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59950 (local address 127.8.18.193:40525)
0810 11:30:41.352419 (+ 856us) server_negotiation.cc:206] Beginning negotiation
0810 11:30:41.352427 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:30:41.352459 (+ 32us) server_negotiation.cc:407] Connection header received
0810 11:30:41.352598 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:30:41.352606 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:30:41.352834 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:30:41.352975 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:30:41.354546 (+ 1571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:41.355610 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:41.356997 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:41.357828 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:41.357955 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:41.362511 (+ 4556us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:30:41.362530 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:30:41.362548 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:30:41.362606 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:30:41.366104 (+ 3498us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:41.367144 (+ 1040us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:41.367153 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:41.367155 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:41.367435 (+ 280us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:41.367918 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:41.367922 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:41.367924 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:41.368544 (+ 620us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:30:41.368830 (+ 286us) server_negotiation.cc:1035] Waiting for connection context
0810 11:30:41.369194 (+ 364us) server_negotiation.cc:299] Negotiation successful
0810 11:30:41.369535 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":706,"thread_start_us":509,"threads_started":1}
Aug 10 11:30:41 dist-test-slave-srgv krb5kdc[11886](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361441, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.18.254@KRBTEST.COM
I20260810 11:30:41.390916 12406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:41.375247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33386 (local address 127.8.18.254:39125)
0810 11:30:41.375490 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0810 11:30:41.375497 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:30:41.375714 (+ 217us) server_negotiation.cc:407] Connection header received
0810 11:30:41.375941 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:30:41.375947 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:30:41.376035 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:30:41.376151 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:30:41.377596 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:41.378369 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:41.379694 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:41.380130 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:41.380277 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:41.384911 (+ 4634us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:30:41.384916 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:30:41.384920 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:30:41.384967 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:30:41.387768 (+ 2801us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:41.388605 (+ 837us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:41.388614 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:41.388621 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:41.388772 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:41.389195 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:41.389202 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:41.389211 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:41.389665 (+ 454us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:30:41.389955 (+ 290us) server_negotiation.cc:1035] Waiting for connection context
0810 11:30:41.390298 (+ 343us) server_negotiation.cc:299] Negotiation successful
0810 11:30:41.390558 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102}
W20260810 11:30:41.391531 11932 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33386
I20260810 11:30:41.392805 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 11997
I20260810 11:30:41.407503 12412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:41.393873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59958 (local address 127.8.18.193:40525)
0810 11:30:41.394154 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0810 11:30:41.394160 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:30:41.394465 (+ 305us) server_negotiation.cc:407] Connection header received
0810 11:30:41.394705 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:30:41.394709 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:30:41.394777 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:30:41.394887 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:30:41.396280 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:41.397009 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:41.398439 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:41.398894 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:41.399059 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:41.401755 (+ 2696us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:30:41.401761 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:30:41.401764 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:30:41.401807 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:30:41.404508 (+ 2701us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:41.405338 (+ 830us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:41.405345 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:41.405354 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:41.405562 (+ 208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:41.405983 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:41.405989 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:41.405993 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:41.406276 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:30:41.406544 (+ 268us) server_negotiation.cc:1035] Waiting for connection context
0810 11:30:41.406892 (+ 348us) server_negotiation.cc:299] Negotiation successful
0810 11:30:41.407146 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111}
I20260810 11:30:41.408249 12124 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:30:41.496955 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 11997
I20260810 11:30:41.515980 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 12136
Aug 10 11:30:41 dist-test-slave-srgv krb5kdc[11886](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361441, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.18.194@KRBTEST.COM
I20260810 11:30:41.535084 12417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:41.517110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46946 (local address 127.8.18.194:44685)
0810 11:30:41.517791 (+ 681us) server_negotiation.cc:206] Beginning negotiation
0810 11:30:41.517799 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:30:41.517828 (+ 29us) server_negotiation.cc:407] Connection header received
0810 11:30:41.517894 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:30:41.517900 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:30:41.518201 (+ 301us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:30:41.518357 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:30:41.520056 (+ 1699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:41.521139 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:41.522569 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:41.523195 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:41.523331 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:41.528072 (+ 4741us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:30:41.528089 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:30:41.528106 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:30:41.528152 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:30:41.531567 (+ 3415us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:41.532524 (+ 957us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:41.532532 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:41.532535 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:41.532701 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:41.533153 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:41.533159 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:41.533163 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:41.533687 (+ 524us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:30:41.534033 (+ 346us) server_negotiation.cc:1035] Waiting for connection context
0810 11:30:41.534391 (+ 358us) server_negotiation.cc:299] Negotiation successful
0810 11:30:41.534685 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":516,"thread_start_us":371,"threads_started":1}
I20260810 11:30:41.536543 12262 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:30:41.622589 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 12136
I20260810 11:30:41.641002 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 12273
Aug 10 11:30:41 dist-test-slave-srgv krb5kdc[11886](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361441, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.18.195@KRBTEST.COM
I20260810 11:30:41.659432 12419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:41.642265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32870 (local address 127.8.18.195:38707)
0810 11:30:41.642930 (+ 665us) server_negotiation.cc:206] Beginning negotiation
0810 11:30:41.642938 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:30:41.642969 (+ 31us) server_negotiation.cc:407] Connection header received
0810 11:30:41.643034 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:30:41.643042 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:30:41.643275 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:30:41.643423 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:30:41.644824 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:41.646067 (+ 1243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:41.647502 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:41.648170 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:41.648294 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:41.653194 (+ 4900us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:30:41.653213 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:30:41.653230 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:30:41.653281 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:30:41.656487 (+ 3206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:41.657267 (+ 780us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:41.657276 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:41.657278 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:41.657507 (+ 229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:41.657940 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:41.657945 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:41.657947 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:41.658367 (+ 420us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:30:41.658629 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0810 11:30:41.658902 (+ 273us) server_negotiation.cc:299] Negotiation successful
0810 11:30:41.659135 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":513,"thread_start_us":346,"threads_started":1}
I20260810 11:30:41.660840 12398 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:30:41.746888 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 12273
I20260810 11:30:41.764400 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 11899
I20260810 11:30:41.765789 11966 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:30:41.874018 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 11899
2026-08-10T11:30:41Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (4250 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786361401922362-8267-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 11:30:41 dist-test-slave-srgv krb5kdc[12425](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 11:30:41 dist-test-slave-srgv krb5kdc[12425](info): set up 2 sockets
Aug 10 11:30:41 dist-test-slave-srgv krb5kdc[12425](info): commencing operation
krb5kdc: starting...
W20260810 11:30:42.996853 8267 mini_kdc.cc:121] Time spent starting KDC: real 1.077s user 0.000s sys 0.024s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786361401922362-8267-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786361401922362-8267-0/krb5kdc/test-user.keytab.
Aug 10 11:30:43 dist-test-slave-srgv krb5kdc[12425](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361443, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T11:30:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T11:30:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.18.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.18.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.18.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.254.keytab.
Entry for principal kudu/127.8.18.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.254.keytab.
I20260810 11:30:43.237529 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786361401922362-8267-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786361401922362-8267-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786361401922362-8267-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.18.254:39185
--webserver_interface=127.8.18.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.254.keytab
--principal=kudu/127.8.18.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.18.212:38827
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.18.254:39185
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786361401922362-8267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786361401922362-8267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786361401922362-8267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:30:43.594159 12438 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:30:43.594501 12438 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:30:43.594708 12438 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:30:43.602440 12438 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 11:30:43.602578 12438 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:30:43.602718 12438 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:30:43.602833 12438 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 11:30:43.602921 12438 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 11:30:43.613600 12438 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.18.212:38827
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786361401922362-8267-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786361401922362-8267-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.18.254:39185
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.254.keytab
--principal=kudu/127.8.18.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.18.254:39185
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.18.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786361401922362-8267-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:30:43.615149 12438 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:30:43.616634 12438 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:30:43.630617 12444 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:30:43.630899 12446 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:30:43.630856 12443 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:30:43.631875 12438 server_base.cc:1034] running on GCE node
I20260810 11:30:43.632755 12438 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:30:43.634405 12438 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:30:43.635625 12438 hybrid_clock.cc:648] HybridClock initialized: now 1786361443635562 us; error 59 us; skew 500 ppm
I20260810 11:30:43.635996 12438 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:30:43 dist-test-slave-srgv krb5kdc[12425](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361443, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:30:43.639731 12438 init.cc:377] Logged in from keytab as kudu/127.8.18.254@KRBTEST.COM (short username kudu)
I20260810 11:30:43.641176 12438 fs_manager.cc:359] Metadata directory not provided
I20260810 11:30:43.641284 12438 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:30:43.641738 12438 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:30:43.645217 12438 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786361401922362-8267-0/minicluster-data/master-0/data/instance:
uuid: "4fbdd77c18624f98b24cf1c7175929ad"
format_stamp: "Formatted at 2026-08-10 11:30:43 on dist-test-slave-srgv"
server_key: "919cf7ef55d5d655250bf5d81513cd39"
server_key_iv: "53d4c9d206a63a21a0fe0101d2fbe8a0"
server_key_version: "encryptionkey@0"
I20260810 11:30:43.646307 12438 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786361401922362-8267-0/minicluster-data/master-0/wal/instance:
uuid: "4fbdd77c18624f98b24cf1c7175929ad"
format_stamp: "Formatted at 2026-08-10 11:30:43 on dist-test-slave-srgv"
server_key: "919cf7ef55d5d655250bf5d81513cd39"
server_key_iv: "53d4c9d206a63a21a0fe0101d2fbe8a0"
server_key_version: "encryptionkey@0"
I20260810 11:30:43.652454 12438 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260810 11:30:43.656672 12452 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:30:43.658330 12438 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 11:30:43.658576 12438 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786361401922362-8267-0/minicluster-data/master-0/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786361401922362-8267-0/minicluster-data/master-0/wal
uuid: "4fbdd77c18624f98b24cf1c7175929ad"
format_stamp: "Formatted at 2026-08-10 11:30:43 on dist-test-slave-srgv"
server_key: "919cf7ef55d5d655250bf5d81513cd39"
server_key_iv: "53d4c9d206a63a21a0fe0101d2fbe8a0"
server_key_version: "encryptionkey@0"
I20260810 11:30:43.658869 12438 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786361401922362-8267-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786361401922362-8267-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786361401922362-8267-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:30:43.672817 12438 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:30:43.677990 12438 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:30:43.678342 12438 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:30:43.698988 12438 rpc_server.cc:307] RPC server started. Bound to: 127.8.18.254:39185
I20260810 11:30:43.699012 12505 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.18.254:39185 every 8 connection(s)
I20260810 11:30:43.700017 12438 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
I20260810 11:30:43.701858 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 12438
I20260810 11:30:43.702085 8267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786361401922362-8267-0/minicluster-data/master-0/wal/instance
I20260810 11:30:43.702771 8267 external_mini_cluster.cc:1428] Setting key bbb6ddc57ffffc7f0f21dff23f39e713
I20260810 11:30:43.705322 12506 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 10 11:30:43 dist-test-slave-srgv krb5kdc[12425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361443, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.18.254@KRBTEST.COM
I20260810 11:30:43.720350 12506 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4fbdd77c18624f98b24cf1c7175929ad: Bootstrap starting.
I20260810 11:30:43.724423 12506 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4fbdd77c18624f98b24cf1c7175929ad: Neither blocks nor log segments found. Creating new log.
I20260810 11:30:43.725718 12506 log.cc:824] T 00000000000000000000000000000000 P 4fbdd77c18624f98b24cf1c7175929ad: Log is configured to *not* fsync() on all Append() calls
I20260810 11:30:43.729512 12506 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4fbdd77c18624f98b24cf1c7175929ad: No bootstrap required, opened a new log
I20260810 11:30:43.729947 12509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:43.704927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48054 (local address 127.8.18.254:39185)
0810 11:30:43.705845 (+ 918us) server_negotiation.cc:206] Beginning negotiation
0810 11:30:43.705859 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 11:30:43.705902 (+ 43us) server_negotiation.cc:407] Connection header received
0810 11:30:43.707714 (+ 1812us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:30:43.707739 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:30:43.708214 (+ 475us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:30:43.708611 (+ 397us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:30:43.710894 (+ 2283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:43.712518 (+ 1624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:43.714641 (+ 2123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:43.715544 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:43.715726 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:43.721175 (+ 5449us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:30:43.721185 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:30:43.721203 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:30:43.721251 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:30:43.724753 (+ 3502us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:43.725713 (+ 960us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:43.725722 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:43.725729 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:43.725972 (+ 243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:43.726531 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:43.726537 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:43.726540 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:43.727234 (+ 694us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:30:43.727553 (+ 319us) server_negotiation.cc:1035] Waiting for connection context
0810 11:30:43.728012 (+ 459us) server_negotiation.cc:299] Negotiation successful
0810 11:30:43.728523 (+ 511us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":699,"thread_start_us":447,"threads_started":1}
I20260810 11:30:43.738340 12506 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4fbdd77c18624f98b24cf1c7175929ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4fbdd77c18624f98b24cf1c7175929ad" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 39185 } }
I20260810 11:30:43.738818 12506 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4fbdd77c18624f98b24cf1c7175929ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:30:43.738957 12506 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4fbdd77c18624f98b24cf1c7175929ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4fbdd77c18624f98b24cf1c7175929ad, State: Initialized, Role: FOLLOWER
I20260810 11:30:43.739724 12506 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4fbdd77c18624f98b24cf1c7175929ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4fbdd77c18624f98b24cf1c7175929ad" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 39185 } }
I20260810 11:30:43.740062 12506 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4fbdd77c18624f98b24cf1c7175929ad [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 11:30:43.740242 12506 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4fbdd77c18624f98b24cf1c7175929ad [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 11:30:43.740450 12506 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4fbdd77c18624f98b24cf1c7175929ad [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:30:43.742962 12506 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4fbdd77c18624f98b24cf1c7175929ad [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4fbdd77c18624f98b24cf1c7175929ad" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 39185 } }
I20260810 11:30:43.743667 12506 leader_election.cc:302] T 00000000000000000000000000000000 P 4fbdd77c18624f98b24cf1c7175929ad [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4fbdd77c18624f98b24cf1c7175929ad; no voters:
I20260810 11:30:43.744401 12506 leader_election.cc:288] T 00000000000000000000000000000000 P 4fbdd77c18624f98b24cf1c7175929ad [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 11:30:43.744578 12512 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4fbdd77c18624f98b24cf1c7175929ad [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:30:43.745376 12512 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4fbdd77c18624f98b24cf1c7175929ad [term 1 LEADER]: Becoming Leader. State: Replica: 4fbdd77c18624f98b24cf1c7175929ad, State: Running, Role: LEADER
I20260810 11:30:43.746017 12512 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4fbdd77c18624f98b24cf1c7175929ad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4fbdd77c18624f98b24cf1c7175929ad" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 39185 } }
I20260810 11:30:43.746896 12506 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4fbdd77c18624f98b24cf1c7175929ad [sys.catalog]: configured and running, proceeding with master startup.
I20260810 11:30:43.749257 12514 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4fbdd77c18624f98b24cf1c7175929ad [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4fbdd77c18624f98b24cf1c7175929ad. Latest consensus state: current_term: 1 leader_uuid: "4fbdd77c18624f98b24cf1c7175929ad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4fbdd77c18624f98b24cf1c7175929ad" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 39185 } } }
I20260810 11:30:43.749797 12514 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4fbdd77c18624f98b24cf1c7175929ad [sys.catalog]: This master's current role is: LEADER
I20260810 11:30:43.751791 12513 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4fbdd77c18624f98b24cf1c7175929ad [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4fbdd77c18624f98b24cf1c7175929ad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4fbdd77c18624f98b24cf1c7175929ad" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 39185 } } }
I20260810 11:30:43.752269 12513 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4fbdd77c18624f98b24cf1c7175929ad [sys.catalog]: This master's current role is: LEADER
I20260810 11:30:43.755052 12520 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 11:30:43.761930 12520 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 11:30:43.772195 12520 catalog_manager.cc:1347] Generated new cluster ID: fefd09225a1d4545996c650364eda966
I20260810 11:30:43.772341 12520 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 11:30:43.783833 12520 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 11:30:43.785352 12520 catalog_manager.cc:1504] Loading token signing keys...
I20260810 11:30:43.796337 12520 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4fbdd77c18624f98b24cf1c7175929ad: Generated new TSK 0
I20260810 11:30:43.797322 12520 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.18.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.18.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.18.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.193.keytab.
Entry for principal kudu/127.8.18.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.193.keytab.
I20260810 11:30:43.863327 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786361401922362-8267-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786361401922362-8267-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786361401922362-8267-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786361401922362-8267-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.18.193:0
--local_ip_for_outbound_sockets=127.8.18.193
--webserver_interface=127.8.18.193
--webserver_port=0
--tserver_master_addrs=127.8.18.254:39185
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.193.keytab
--principal=kudu/127.8.18.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.18.212:38827
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786361401922362-8267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786361401922362-8267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786361401922362-8267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:30:44.232002 12533 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:30:44.232401 12533 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:30:44.232568 12533 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:30:44.240456 12533 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:30:44.240634 12533 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:30:44.240839 12533 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.18.193
I20260810 11:30:44.251621 12533 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.18.212:38827
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786361401922362-8267-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786361401922362-8267-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.193.keytab
--principal=kudu/127.8.18.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.18.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786361401922362-8267-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.18.193
--webserver_port=0
--tserver_master_addrs=127.8.18.254:39185
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.18.193
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786361401922362-8267-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:30:44.253172 12533 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:30:44.254803 12533 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:30:44.269636 12539 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:30:44.269801 12538 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:30:44.269840 12541 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:30:44.270936 12533 server_base.cc:1034] running on GCE node
I20260810 11:30:44.271697 12533 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:30:44.272872 12533 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:30:44.274140 12533 hybrid_clock.cc:648] HybridClock initialized: now 1786361444274125 us; error 78 us; skew 500 ppm
I20260810 11:30:44.274530 12533 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:30:44 dist-test-slave-srgv krb5kdc[12425](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361444, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:30:44.278244 12533 init.cc:377] Logged in from keytab as kudu/127.8.18.193@KRBTEST.COM (short username kudu)
I20260810 11:30:44.279528 12533 fs_manager.cc:359] Metadata directory not provided
I20260810 11:30:44.279640 12533 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:30:44.279929 12533 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:30:44.283336 12533 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786361401922362-8267-0/minicluster-data/ts-0/data/instance:
uuid: "cea165d06fe944b68f3f05c704f508a8"
format_stamp: "Formatted at 2026-08-10 11:30:44 on dist-test-slave-srgv"
server_key: "a7f18330b44614287bd7ef79f238b240"
server_key_iv: "5f025c5782f38ee8c72cb384279b689b"
server_key_version: "encryptionkey@0"
I20260810 11:30:44.284319 12533 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786361401922362-8267-0/minicluster-data/ts-0/wal/instance:
uuid: "cea165d06fe944b68f3f05c704f508a8"
format_stamp: "Formatted at 2026-08-10 11:30:44 on dist-test-slave-srgv"
server_key: "a7f18330b44614287bd7ef79f238b240"
server_key_iv: "5f025c5782f38ee8c72cb384279b689b"
server_key_version: "encryptionkey@0"
I20260810 11:30:44.290212 12533 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260810 11:30:44.294445 12547 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:30:44.296078 12533 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 11:30:44.296326 12533 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786361401922362-8267-0/minicluster-data/ts-0/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786361401922362-8267-0/minicluster-data/ts-0/wal
uuid: "cea165d06fe944b68f3f05c704f508a8"
format_stamp: "Formatted at 2026-08-10 11:30:44 on dist-test-slave-srgv"
server_key: "a7f18330b44614287bd7ef79f238b240"
server_key_iv: "5f025c5782f38ee8c72cb384279b689b"
server_key_version: "encryptionkey@0"
I20260810 11:30:44.296604 12533 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786361401922362-8267-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786361401922362-8267-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786361401922362-8267-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:30:44.313895 12533 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:30:44.319162 12533 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:30:44.319507 12533 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:30:44.320668 12533 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:30:44.322463 12533 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:30:44.322609 12533 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:30:44.322765 12533 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:30:44.322860 12533 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:30:44.359978 12533 rpc_server.cc:307] RPC server started. Bound to: 127.8.18.193:38439
I20260810 11:30:44.360116 12662 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.18.193:38439 every 8 connection(s)
I20260810 11:30:44.360989 12533 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786361401922362-8267-0/minicluster-data/ts-0/data/info.pb
I20260810 11:30:44.362239 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 12533
I20260810 11:30:44.362470 8267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786361401922362-8267-0/minicluster-data/ts-0/wal/instance
I20260810 11:30:44.363358 8267 external_mini_cluster.cc:1428] Setting key 8ddba91a9e6c3e0251fdc553d812986a
Aug 10 11:30:44 dist-test-slave-srgv krb5kdc[12425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361444, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.193@KRBTEST.COM for kudu/127.8.18.254@KRBTEST.COM
I20260810 11:30:44.384768 12665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:44.364296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.193:53887 (local address 127.8.18.254:39185)
0810 11:30:44.364828 (+ 532us) server_negotiation.cc:206] Beginning negotiation
0810 11:30:44.364836 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:30:44.365987 (+ 1151us) server_negotiation.cc:407] Connection header received
0810 11:30:44.367983 (+ 1996us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:30:44.367989 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:30:44.368081 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:30:44.368215 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:30:44.370020 (+ 1805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:44.370765 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:44.372546 (+ 1781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:44.372985 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:44.373100 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:44.378057 (+ 4957us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:30:44.378063 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:30:44.378067 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:30:44.378108 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:30:44.380961 (+ 2853us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:44.382053 (+ 1092us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:44.382058 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:44.382060 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:44.382202 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:44.382732 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:44.382736 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:44.382738 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:44.383055 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:30:44.383233 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0810 11:30:44.384105 (+ 872us) server_negotiation.cc:299] Negotiation successful
0810 11:30:44.384369 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":404,"thread_start_us":286,"threads_started":1}
WARNING: no policy specified for kudu/127.8.18.194@KRBTEST.COM; defaulting to no policy
I20260810 11:30:44.385988 12666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:44.364822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:39185 (local address 127.8.18.193:53887)
0810 11:30:44.365744 (+ 922us) negotiation.cc:107] Waiting for socket to connect
0810 11:30:44.365807 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0810 11:30:44.367659 (+ 1852us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:30:44.368502 (+ 843us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:30:44.368512 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:30:44.369101 (+ 589us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:30:44.369813 (+ 712us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:30:44.369834 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:30:44.370954 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:30:44.370959 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:30:44.372340 (+ 1381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:30:44.372361 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:30:44.373279 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:30:44.373289 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:30:44.373685 (+ 396us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:44.374521 (+ 836us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:30:44.374558 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:30:44.377743 (+ 3185us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:30:44.381192 (+ 3449us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:30:44.381204 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:30:44.381226 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:30:44.381866 (+ 640us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:30:44.382341 (+ 475us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:30:44.382347 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:30:44.382350 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:30:44.382578 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:30:44.383218 (+ 640us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:30:44.383228 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:30:44.383659 (+ 431us) client_negotiation.cc:769] Sending connection context
0810 11:30:44.383952 (+ 293us) client_negotiation.cc:240] Negotiation successful
0810 11:30:44.384590 (+ 638us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":614,"thread_start_us":359,"threads_started":1}
I20260810 11:30:44.388055 12663 heartbeater.cc:347] Connected to a master server at 127.8.18.254:39185
I20260810 11:30:44.388430 12663 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.18.194@KRBTEST.COM" created.
I20260810 11:30:44.389287 12663 heartbeater.cc:511] Master 127.8.18.254:39185 requested a full tablet report, sending...
I20260810 11:30:44.392140 12470 ts_manager.cc:194] Registered new tserver with Master: cea165d06fe944b68f3f05c704f508a8 (127.8.18.193:38439)
I20260810 11:30:44.394447 12470 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.18.193@KRBTEST.COM'} at 127.8.18.193:53887
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.18.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.194.keytab.
Entry for principal kudu/127.8.18.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.194.keytab.
I20260810 11:30:44.422617 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786361401922362-8267-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786361401922362-8267-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786361401922362-8267-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786361401922362-8267-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.18.194:0
--local_ip_for_outbound_sockets=127.8.18.194
--webserver_interface=127.8.18.194
--webserver_port=0
--tserver_master_addrs=127.8.18.254:39185
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.194.keytab
--principal=kudu/127.8.18.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.18.212:38827
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786361401922362-8267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786361401922362-8267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786361401922362-8267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:30:44.778432 12669 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:30:44.778865 12669 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:30:44.779040 12669 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:30:44.786746 12669 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:30:44.786943 12669 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:30:44.787130 12669 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.18.194
I20260810 11:30:44.798413 12669 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.18.212:38827
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786361401922362-8267-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786361401922362-8267-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.194.keytab
--principal=kudu/127.8.18.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.18.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786361401922362-8267-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.18.194
--webserver_port=0
--tserver_master_addrs=127.8.18.254:39185
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.18.194
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786361401922362-8267-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:30:44.800022 12669 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:30:44.801828 12669 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:30:44.816915 12675 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:30:44.817047 12677 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:30:44.817497 12674 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:30:44.817978 12669 server_base.cc:1034] running on GCE node
I20260810 11:30:44.818758 12669 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:30:44.819941 12669 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:30:44.821376 12669 hybrid_clock.cc:648] HybridClock initialized: now 1786361444821200 us; error 105 us; skew 500 ppm
I20260810 11:30:44.821878 12669 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:30:44 dist-test-slave-srgv krb5kdc[12425](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361444, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:30:44.825691 12669 init.cc:377] Logged in from keytab as kudu/127.8.18.194@KRBTEST.COM (short username kudu)
I20260810 11:30:44.827293 12669 fs_manager.cc:359] Metadata directory not provided
I20260810 11:30:44.827451 12669 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:30:44.827816 12669 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:30:44.831225 12669 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786361401922362-8267-0/minicluster-data/ts-1/data/instance:
uuid: "9e5efa7a99fe42d287bb58c8fc0470ea"
format_stamp: "Formatted at 2026-08-10 11:30:44 on dist-test-slave-srgv"
server_key: "9924993f8cbbbdcbe97e018593a4405b"
server_key_iv: "ce7ca2567dc6b1791ca0f9dccc7f1a93"
server_key_version: "encryptionkey@0"
I20260810 11:30:44.832175 12669 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786361401922362-8267-0/minicluster-data/ts-1/wal/instance:
uuid: "9e5efa7a99fe42d287bb58c8fc0470ea"
format_stamp: "Formatted at 2026-08-10 11:30:44 on dist-test-slave-srgv"
server_key: "9924993f8cbbbdcbe97e018593a4405b"
server_key_iv: "ce7ca2567dc6b1791ca0f9dccc7f1a93"
server_key_version: "encryptionkey@0"
I20260810 11:30:44.838420 12669 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260810 11:30:44.842914 12683 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:30:44.844692 12669 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.002s
I20260810 11:30:44.844965 12669 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786361401922362-8267-0/minicluster-data/ts-1/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786361401922362-8267-0/minicluster-data/ts-1/wal
uuid: "9e5efa7a99fe42d287bb58c8fc0470ea"
format_stamp: "Formatted at 2026-08-10 11:30:44 on dist-test-slave-srgv"
server_key: "9924993f8cbbbdcbe97e018593a4405b"
server_key_iv: "ce7ca2567dc6b1791ca0f9dccc7f1a93"
server_key_version: "encryptionkey@0"
I20260810 11:30:44.845252 12669 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786361401922362-8267-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786361401922362-8267-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786361401922362-8267-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:30:44.867345 12669 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:30:44.872686 12669 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:30:44.873027 12669 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:30:44.874228 12669 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:30:44.876031 12669 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:30:44.876232 12669 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:30:44.876386 12669 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:30:44.876479 12669 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:30:44.914495 12669 rpc_server.cc:307] RPC server started. Bound to: 127.8.18.194:38403
I20260810 11:30:44.914551 12798 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.18.194:38403 every 8 connection(s)
I20260810 11:30:44.915488 12669 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786361401922362-8267-0/minicluster-data/ts-1/data/info.pb
I20260810 11:30:44.918062 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 12669
I20260810 11:30:44.918311 8267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786361401922362-8267-0/minicluster-data/ts-1/wal/instance
I20260810 11:30:44.919024 8267 external_mini_cluster.cc:1428] Setting key b30eb315a69197e1c3542bafb98e6a71
Aug 10 11:30:44 dist-test-slave-srgv krb5kdc[12425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361444, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.194@KRBTEST.COM for kudu/127.8.18.254@KRBTEST.COM
I20260810 11:30:44.941071 12801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:44.918360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.194:58089 (local address 127.8.18.254:39185)
0810 11:30:44.918902 (+ 542us) server_negotiation.cc:206] Beginning negotiation
0810 11:30:44.918910 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:30:44.920136 (+ 1226us) server_negotiation.cc:407] Connection header received
0810 11:30:44.922415 (+ 2279us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:30:44.922424 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:30:44.922529 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:30:44.922661 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:30:44.924639 (+ 1978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:44.925933 (+ 1294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:44.927702 (+ 1769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:44.928471 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:44.928581 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:44.933620 (+ 5039us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:30:44.933626 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:30:44.933629 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:30:44.933673 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:30:44.936952 (+ 3279us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:44.938143 (+ 1191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:44.938149 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:44.938152 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:44.938333 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:44.938914 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:44.938919 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:44.938922 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:44.939248 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:30:44.939443 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0810 11:30:44.940346 (+ 903us) server_negotiation.cc:299] Negotiation successful
0810 11:30:44.940640 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":400,"thread_start_us":263,"threads_started":1}
WARNING: no policy specified for kudu/127.8.18.195@KRBTEST.COM; defaulting to no policy
I20260810 11:30:44.942314 12802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:44.918978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:39185 (local address 127.8.18.194:58089)
0810 11:30:44.919939 (+ 961us) negotiation.cc:107] Waiting for socket to connect
0810 11:30:44.920005 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0810 11:30:44.922070 (+ 2065us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:30:44.922903 (+ 833us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:30:44.922912 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:30:44.923664 (+ 752us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:30:44.924388 (+ 724us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:30:44.924409 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:30:44.926114 (+ 1705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:30:44.926118 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:30:44.927503 (+ 1385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:30:44.927517 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:30:44.928751 (+ 1234us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:30:44.928762 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:30:44.929120 (+ 358us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:44.930024 (+ 904us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:30:44.930061 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:30:44.933253 (+ 3192us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:30:44.937209 (+ 3956us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:30:44.937221 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:30:44.937252 (+ 31us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:30:44.937938 (+ 686us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:30:44.938492 (+ 554us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:30:44.938498 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:30:44.938503 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:30:44.938746 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:30:44.939446 (+ 700us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:30:44.939457 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:30:44.939891 (+ 434us) client_negotiation.cc:769] Sending connection context
0810 11:30:44.940218 (+ 327us) client_negotiation.cc:240] Negotiation successful
0810 11:30:44.940869 (+ 651us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":657,"thread_start_us":447,"threads_started":1}
I20260810 11:30:44.944207 12799 heartbeater.cc:347] Connected to a master server at 127.8.18.254:39185
I20260810 11:30:44.944676 12799 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.18.195@KRBTEST.COM" created.
I20260810 11:30:44.945695 12799 heartbeater.cc:511] Master 127.8.18.254:39185 requested a full tablet report, sending...
I20260810 11:30:44.947827 12470 ts_manager.cc:194] Registered new tserver with Master: 9e5efa7a99fe42d287bb58c8fc0470ea (127.8.18.194:38403)
I20260810 11:30:44.948846 12470 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.18.194@KRBTEST.COM'} at 127.8.18.194:58089
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.18.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.195.keytab.
Entry for principal kudu/127.8.18.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.195.keytab.
I20260810 11:30:44.977831 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786361401922362-8267-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786361401922362-8267-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786361401922362-8267-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786361401922362-8267-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.18.195:0
--local_ip_for_outbound_sockets=127.8.18.195
--webserver_interface=127.8.18.195
--webserver_port=0
--tserver_master_addrs=127.8.18.254:39185
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.195.keytab
--principal=kudu/127.8.18.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.18.212:38827
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786361401922362-8267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786361401922362-8267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786361401922362-8267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:30:45.339658 12805 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:30:45.340057 12805 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:30:45.340251 12805 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:30:45.348291 12805 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:30:45.348485 12805 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:30:45.348673 12805 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.18.195
I20260810 11:30:45.359671 12805 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.18.212:38827
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786361401922362-8267-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786361401922362-8267-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.195.keytab
--principal=kudu/127.8.18.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.18.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786361401922362-8267-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.18.195
--webserver_port=0
--tserver_master_addrs=127.8.18.254:39185
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.18.195
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786361401922362-8267-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:30:45.361131 12805 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:30:45.362664 12805 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:30:45.378963 12810 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:30:45.378962 12811 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:30:45.378960 12813 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:30:45.379494 12805 server_base.cc:1034] running on GCE node
I20260810 11:30:45.380393 12805 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:30:45.381589 12805 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:30:45.382845 12805 hybrid_clock.cc:648] HybridClock initialized: now 1786361445382774 us; error 75 us; skew 500 ppm
I20260810 11:30:45.383239 12805 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:30:45 dist-test-slave-srgv krb5kdc[12425](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361445, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:30:45.387104 12805 init.cc:377] Logged in from keytab as kudu/127.8.18.195@KRBTEST.COM (short username kudu)
I20260810 11:30:45.388597 12805 fs_manager.cc:359] Metadata directory not provided
I20260810 11:30:45.388725 12805 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:30:45.389035 12805 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:30:45.392521 12805 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786361401922362-8267-0/minicluster-data/ts-2/data/instance:
uuid: "977bcfa24bb04ead85957eef52f199f5"
format_stamp: "Formatted at 2026-08-10 11:30:45 on dist-test-slave-srgv"
server_key: "615309ea9febaaf01b1ab3193da23b31"
server_key_iv: "29df2f0e67c7ea9092123dcb0cf062ff"
server_key_version: "encryptionkey@0"
I20260810 11:30:45.393535 12805 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786361401922362-8267-0/minicluster-data/ts-2/wal/instance:
uuid: "977bcfa24bb04ead85957eef52f199f5"
format_stamp: "Formatted at 2026-08-10 11:30:45 on dist-test-slave-srgv"
server_key: "615309ea9febaaf01b1ab3193da23b31"
server_key_iv: "29df2f0e67c7ea9092123dcb0cf062ff"
server_key_version: "encryptionkey@0"
I20260810 11:30:45.398480 12663 heartbeater.cc:503] Master 127.8.18.254:39185 was elected leader, sending a full tablet report...
I20260810 11:30:45.399441 12805 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.002s
I20260810 11:30:45.404259 12819 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:30:45.406101 12805 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260810 11:30:45.406282 12805 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786361401922362-8267-0/minicluster-data/ts-2/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786361401922362-8267-0/minicluster-data/ts-2/wal
uuid: "977bcfa24bb04ead85957eef52f199f5"
format_stamp: "Formatted at 2026-08-10 11:30:45 on dist-test-slave-srgv"
server_key: "615309ea9febaaf01b1ab3193da23b31"
server_key_iv: "29df2f0e67c7ea9092123dcb0cf062ff"
server_key_version: "encryptionkey@0"
I20260810 11:30:45.406514 12805 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786361401922362-8267-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786361401922362-8267-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786361401922362-8267-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:30:45.420854 12805 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:30:45.425954 12805 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:30:45.426223 12805 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:30:45.427348 12805 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:30:45.429011 12805 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:30:45.429116 12805 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:30:45.429196 12805 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:30:45.429273 12805 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:30:45.467530 12805 rpc_server.cc:307] RPC server started. Bound to: 127.8.18.195:36797
I20260810 11:30:45.467589 12934 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.18.195:36797 every 8 connection(s)
I20260810 11:30:45.468538 12805 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786361401922362-8267-0/minicluster-data/ts-2/data/info.pb
I20260810 11:30:45.473483 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 12805
I20260810 11:30:45.473678 8267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786361401922362-8267-0/minicluster-data/ts-2/wal/instance
I20260810 11:30:45.474179 8267 external_mini_cluster.cc:1428] Setting key 4b7923c0b5c180da313099331788111b
Aug 10 11:30:45 dist-test-slave-srgv krb5kdc[12425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361445, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.195@KRBTEST.COM for kudu/127.8.18.254@KRBTEST.COM
I20260810 11:30:45.491410 12937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:45.471223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.195:36621 (local address 127.8.18.254:39185)
0810 11:30:45.471777 (+ 554us) server_negotiation.cc:206] Beginning negotiation
0810 11:30:45.471783 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:30:45.472685 (+ 902us) server_negotiation.cc:407] Connection header received
0810 11:30:45.474975 (+ 2290us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:30:45.474981 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:30:45.475089 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:30:45.475230 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:30:45.477025 (+ 1795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:45.478066 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:45.479817 (+ 1751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:45.480282 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:45.480439 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:45.484649 (+ 4210us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:30:45.484655 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:30:45.484658 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:30:45.484700 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:30:45.487685 (+ 2985us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:45.488691 (+ 1006us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:45.488699 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:45.488703 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:45.488872 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:45.489385 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:45.489392 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:45.489395 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:45.489753 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:30:45.489935 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0810 11:30:45.490751 (+ 816us) server_negotiation.cc:299] Negotiation successful
0810 11:30:45.491013 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":437,"thread_start_us":305,"threads_started":1}
I20260810 11:30:45.492540 12938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:45.471665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:39185 (local address 127.8.18.195:36621)
0810 11:30:45.472485 (+ 820us) negotiation.cc:107] Waiting for socket to connect
0810 11:30:45.472538 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0810 11:30:45.474606 (+ 2068us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:30:45.475516 (+ 910us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:30:45.475527 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:30:45.476133 (+ 606us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:30:45.476802 (+ 669us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:30:45.476822 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:30:45.478270 (+ 1448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:30:45.478274 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:30:45.479633 (+ 1359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:30:45.479648 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:30:45.480573 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:30:45.480581 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:30:45.480843 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:45.481683 (+ 840us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:30:45.481716 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:30:45.484311 (+ 2595us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:30:45.487922 (+ 3611us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:30:45.487934 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:30:45.487964 (+ 30us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:30:45.488533 (+ 569us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:30:45.489013 (+ 480us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:30:45.489019 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:30:45.489023 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:30:45.489256 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:30:45.489906 (+ 650us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:30:45.489915 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:30:45.490342 (+ 427us) client_negotiation.cc:769] Sending connection context
0810 11:30:45.490649 (+ 307us) client_negotiation.cc:240] Negotiation successful
0810 11:30:45.491256 (+ 607us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":585,"thread_start_us":381,"threads_started":1}
I20260810 11:30:45.494234 12935 heartbeater.cc:347] Connected to a master server at 127.8.18.254:39185
I20260810 11:30:45.494601 12935 heartbeater.cc:464] Registering TS with master...
I20260810 11:30:45.495318 12935 heartbeater.cc:511] Master 127.8.18.254:39185 requested a full tablet report, sending...
I20260810 11:30:45.497020 12470 ts_manager.cc:194] Registered new tserver with Master: 977bcfa24bb04ead85957eef52f199f5 (127.8.18.195:36797)
I20260810 11:30:45.498008 12470 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.18.195@KRBTEST.COM'} at 127.8.18.195:36621
I20260810 11:30:45.503436 8267 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 11:30:45.524282 12937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:45.508610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48062 (local address 127.8.18.254:39185)
0810 11:30:45.508801 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0810 11:30:45.508806 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:30:45.509321 (+ 515us) server_negotiation.cc:407] Connection header received
0810 11:30:45.509589 (+ 268us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:30:45.509595 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:30:45.509674 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:30:45.509823 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:30:45.511927 (+ 2104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:45.512683 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:45.514216 (+ 1533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:45.514655 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:45.514832 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:45.517553 (+ 2721us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:30:45.517558 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:30:45.517562 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:30:45.517606 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:30:45.520967 (+ 3361us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:45.521938 (+ 971us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:45.521944 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:45.521946 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:45.522152 (+ 206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:45.522571 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:45.522574 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:45.522577 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:45.522964 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:30:45.523206 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0810 11:30:45.523566 (+ 360us) server_negotiation.cc:299] Negotiation successful
0810 11:30:45.523797 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260810 11:30:45.530057 12470 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48062:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 11:30:45.537563 12470 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 11:30:45.565820 12948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:45.552126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.194:38403 (local address 127.0.0.1:45670)
0810 11:30:45.554204 (+ 2078us) negotiation.cc:107] Waiting for socket to connect
0810 11:30:45.554266 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0810 11:30:45.554522 (+ 256us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:30:45.555843 (+ 1321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:30:45.555855 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:30:45.555900 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:30:45.556515 (+ 615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:30:45.556551 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:30:45.559411 (+ 2860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:30:45.559422 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:30:45.562522 (+ 3100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:30:45.562537 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:30:45.564509 (+ 1972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:30:45.564517 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:30:45.564747 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:45.564810 (+ 63us) client_negotiation.cc:769] Sending connection context
0810 11:30:45.564976 (+ 166us) client_negotiation.cc:240] Negotiation successful
0810 11:30:45.565227 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1555,"mutex_wait_us":52,"spinlock_wait_cycles":11008,"thread_start_us":1340,"threads_started":1}
I20260810 11:30:45.566524 12950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:45.552397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41920 (local address 127.8.18.195:36797)
0810 11:30:45.553582 (+ 1185us) server_negotiation.cc:206] Beginning negotiation
0810 11:30:45.553591 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 11:30:45.554983 (+ 1392us) server_negotiation.cc:407] Connection header received
0810 11:30:45.555215 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:30:45.555224 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:30:45.555453 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:30:45.555678 (+ 225us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:30:45.556965 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:45.559141 (+ 2176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:45.563436 (+ 4295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:45.565046 (+ 1610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:45.565176 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:45.565392 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0810 11:30:45.565977 (+ 585us) server_negotiation.cc:299] Negotiation successful
0810 11:30:45.566164 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1020,"thread_start_us":841,"threads_started":1}
I20260810 11:30:45.566933 12949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:45.552510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45670 (local address 127.8.18.194:38403)
0810 11:30:45.553216 (+ 706us) server_negotiation.cc:206] Beginning negotiation
0810 11:30:45.553228 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 11:30:45.554777 (+ 1549us) server_negotiation.cc:407] Connection header received
0810 11:30:45.554890 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:30:45.554898 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:30:45.555207 (+ 309us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:30:45.555456 (+ 249us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:30:45.556816 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:45.559140 (+ 2324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:45.562726 (+ 3586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:45.564224 (+ 1498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:45.564434 (+ 210us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:45.564700 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0810 11:30:45.565560 (+ 860us) server_negotiation.cc:299] Negotiation successful
0810 11:30:45.565770 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":522,"thread_start_us":373,"threads_started":1}
I20260810 11:30:45.567044 12951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:45.552417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.195:36797 (local address 127.0.0.1:41920)
0810 11:30:45.554703 (+ 2286us) negotiation.cc:107] Waiting for socket to connect
0810 11:30:45.554718 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 11:30:45.554882 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:30:45.556068 (+ 1186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:30:45.556076 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:30:45.556139 (+ 63us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:30:45.556693 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:30:45.556707 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:30:45.559419 (+ 2712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:30:45.559429 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:30:45.563216 (+ 3787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:30:45.563237 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:30:45.565479 (+ 2242us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:30:45.565491 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:30:45.565732 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:45.565779 (+ 47us) client_negotiation.cc:769] Sending connection context
0810 11:30:45.565892 (+ 113us) client_negotiation.cc:240] Negotiation successful
0810 11:30:45.566330 (+ 438us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2189,"mutex_wait_us":1473,"spinlock_wait_cycles":7936,"thread_start_us":1996,"threads_started":1}
I20260810 11:30:45.570154 12952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:45.554874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.193:38439 (local address 127.0.0.1:37276)
0810 11:30:45.560531 (+ 5657us) negotiation.cc:107] Waiting for socket to connect
0810 11:30:45.560553 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 11:30:45.560725 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:30:45.561819 (+ 1094us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:30:45.561828 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:30:45.561854 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:30:45.562395 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:30:45.562414 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:30:45.564357 (+ 1943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:30:45.564363 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:30:45.567603 (+ 3240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:30:45.567622 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:30:45.569286 (+ 1664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:30:45.569293 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:30:45.569502 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:45.569548 (+ 46us) client_negotiation.cc:769] Sending connection context
0810 11:30:45.569658 (+ 110us) client_negotiation.cc:240] Negotiation successful
0810 11:30:45.569807 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5486,"thread_start_us":5333,"threads_started":1}
I20260810 11:30:45.571377 12870 tablet_service.cc:1467] Processing CreateTablet for tablet 49f8c6571a09487cb520ff8dd8301cf9 (DEFAULT_TABLE table=test-table [id=c68eb4f1ef7140928ce896fc53ae2d90]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:30:45.572085 12954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:45.555291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37276 (local address 127.8.18.193:38439)
0810 11:30:45.560040 (+ 4749us) server_negotiation.cc:206] Beginning negotiation
0810 11:30:45.560047 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:30:45.560698 (+ 651us) server_negotiation.cc:407] Connection header received
0810 11:30:45.560937 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:30:45.560948 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:30:45.561273 (+ 325us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:30:45.561534 (+ 261us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:30:45.562669 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:45.564161 (+ 1492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:45.567833 (+ 3672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:45.569032 (+ 1199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:45.569167 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:45.569348 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0810 11:30:45.569743 (+ 395us) server_negotiation.cc:299] Negotiation successful
0810 11:30:45.569935 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4551,"thread_start_us":4372,"threads_started":1}
I20260810 11:30:45.572889 12870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 49f8c6571a09487cb520ff8dd8301cf9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:30:45.572501 12733 tablet_service.cc:1467] Processing CreateTablet for tablet 49f8c6571a09487cb520ff8dd8301cf9 (DEFAULT_TABLE table=test-table [id=c68eb4f1ef7140928ce896fc53ae2d90]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:30:45.573997 12733 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 49f8c6571a09487cb520ff8dd8301cf9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:30:45.576465 12598 tablet_service.cc:1467] Processing CreateTablet for tablet 49f8c6571a09487cb520ff8dd8301cf9 (DEFAULT_TABLE table=test-table [id=c68eb4f1ef7140928ce896fc53ae2d90]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:30:45.578377 12598 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 49f8c6571a09487cb520ff8dd8301cf9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:30:45.588088 12955 tablet_bootstrap.cc:492] T 49f8c6571a09487cb520ff8dd8301cf9 P 9e5efa7a99fe42d287bb58c8fc0470ea: Bootstrap starting.
I20260810 11:30:45.589246 12957 tablet_bootstrap.cc:492] T 49f8c6571a09487cb520ff8dd8301cf9 P 977bcfa24bb04ead85957eef52f199f5: Bootstrap starting.
I20260810 11:30:45.593727 12957 tablet_bootstrap.cc:654] T 49f8c6571a09487cb520ff8dd8301cf9 P 977bcfa24bb04ead85957eef52f199f5: Neither blocks nor log segments found. Creating new log.
I20260810 11:30:45.593726 12955 tablet_bootstrap.cc:654] T 49f8c6571a09487cb520ff8dd8301cf9 P 9e5efa7a99fe42d287bb58c8fc0470ea: Neither blocks nor log segments found. Creating new log.
I20260810 11:30:45.593834 12958 tablet_bootstrap.cc:492] T 49f8c6571a09487cb520ff8dd8301cf9 P cea165d06fe944b68f3f05c704f508a8: Bootstrap starting.
I20260810 11:30:45.595276 12957 log.cc:824] T 49f8c6571a09487cb520ff8dd8301cf9 P 977bcfa24bb04ead85957eef52f199f5: Log is configured to *not* fsync() on all Append() calls
I20260810 11:30:45.596804 12955 log.cc:824] T 49f8c6571a09487cb520ff8dd8301cf9 P 9e5efa7a99fe42d287bb58c8fc0470ea: Log is configured to *not* fsync() on all Append() calls
I20260810 11:30:45.599421 12957 tablet_bootstrap.cc:492] T 49f8c6571a09487cb520ff8dd8301cf9 P 977bcfa24bb04ead85957eef52f199f5: No bootstrap required, opened a new log
I20260810 11:30:45.599787 12957 ts_tablet_manager.cc:1397] T 49f8c6571a09487cb520ff8dd8301cf9 P 977bcfa24bb04ead85957eef52f199f5: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.004s
I20260810 11:30:45.599895 12958 tablet_bootstrap.cc:654] T 49f8c6571a09487cb520ff8dd8301cf9 P cea165d06fe944b68f3f05c704f508a8: Neither blocks nor log segments found. Creating new log.
I20260810 11:30:45.600612 12955 tablet_bootstrap.cc:492] T 49f8c6571a09487cb520ff8dd8301cf9 P 9e5efa7a99fe42d287bb58c8fc0470ea: No bootstrap required, opened a new log
I20260810 11:30:45.600925 12955 ts_tablet_manager.cc:1397] T 49f8c6571a09487cb520ff8dd8301cf9 P 9e5efa7a99fe42d287bb58c8fc0470ea: Time spent bootstrapping tablet: real 0.013s user 0.001s sys 0.008s
I20260810 11:30:45.601315 12958 log.cc:824] T 49f8c6571a09487cb520ff8dd8301cf9 P cea165d06fe944b68f3f05c704f508a8: Log is configured to *not* fsync() on all Append() calls
I20260810 11:30:45.604647 12958 tablet_bootstrap.cc:492] T 49f8c6571a09487cb520ff8dd8301cf9 P cea165d06fe944b68f3f05c704f508a8: No bootstrap required, opened a new log
I20260810 11:30:45.605016 12958 ts_tablet_manager.cc:1397] T 49f8c6571a09487cb520ff8dd8301cf9 P cea165d06fe944b68f3f05c704f508a8: Time spent bootstrapping tablet: real 0.012s user 0.005s sys 0.003s
I20260810 11:30:45.612051 12957 raft_consensus.cc:348] T 49f8c6571a09487cb520ff8dd8301cf9 P 977bcfa24bb04ead85957eef52f199f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "977bcfa24bb04ead85957eef52f199f5" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 36797 } } peers { permanent_uuid: "9e5efa7a99fe42d287bb58c8fc0470ea" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 38403 } } peers { permanent_uuid: "cea165d06fe944b68f3f05c704f508a8" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 38439 } }
I20260810 11:30:45.612663 12957 raft_consensus.cc:374] T 49f8c6571a09487cb520ff8dd8301cf9 P 977bcfa24bb04ead85957eef52f199f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:30:45.612785 12957 raft_consensus.cc:729] T 49f8c6571a09487cb520ff8dd8301cf9 P 977bcfa24bb04ead85957eef52f199f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 977bcfa24bb04ead85957eef52f199f5, State: Initialized, Role: FOLLOWER
I20260810 11:30:45.613662 12957 consensus_queue.cc:260] T 49f8c6571a09487cb520ff8dd8301cf9 P 977bcfa24bb04ead85957eef52f199f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "977bcfa24bb04ead85957eef52f199f5" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 36797 } } peers { permanent_uuid: "9e5efa7a99fe42d287bb58c8fc0470ea" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 38403 } } peers { permanent_uuid: "cea165d06fe944b68f3f05c704f508a8" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 38439 } }
I20260810 11:30:45.613382 12955 raft_consensus.cc:348] T 49f8c6571a09487cb520ff8dd8301cf9 P 9e5efa7a99fe42d287bb58c8fc0470ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "977bcfa24bb04ead85957eef52f199f5" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 36797 } } peers { permanent_uuid: "9e5efa7a99fe42d287bb58c8fc0470ea" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 38403 } } peers { permanent_uuid: "cea165d06fe944b68f3f05c704f508a8" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 38439 } }
I20260810 11:30:45.614060 12955 raft_consensus.cc:374] T 49f8c6571a09487cb520ff8dd8301cf9 P 9e5efa7a99fe42d287bb58c8fc0470ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:30:45.614193 12955 raft_consensus.cc:729] T 49f8c6571a09487cb520ff8dd8301cf9 P 9e5efa7a99fe42d287bb58c8fc0470ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e5efa7a99fe42d287bb58c8fc0470ea, State: Initialized, Role: FOLLOWER
I20260810 11:30:45.615120 12935 heartbeater.cc:503] Master 127.8.18.254:39185 was elected leader, sending a full tablet report...
I20260810 11:30:45.615628 12958 raft_consensus.cc:348] T 49f8c6571a09487cb520ff8dd8301cf9 P cea165d06fe944b68f3f05c704f508a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "977bcfa24bb04ead85957eef52f199f5" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 36797 } } peers { permanent_uuid: "9e5efa7a99fe42d287bb58c8fc0470ea" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 38403 } } peers { permanent_uuid: "cea165d06fe944b68f3f05c704f508a8" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 38439 } }
I20260810 11:30:45.615891 12957 ts_tablet_manager.cc:1428] T 49f8c6571a09487cb520ff8dd8301cf9 P 977bcfa24bb04ead85957eef52f199f5: Time spent starting tablet: real 0.016s user 0.014s sys 0.003s
I20260810 11:30:45.616077 12958 raft_consensus.cc:374] T 49f8c6571a09487cb520ff8dd8301cf9 P cea165d06fe944b68f3f05c704f508a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:30:45.616226 12958 raft_consensus.cc:729] T 49f8c6571a09487cb520ff8dd8301cf9 P cea165d06fe944b68f3f05c704f508a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cea165d06fe944b68f3f05c704f508a8, State: Initialized, Role: FOLLOWER
I20260810 11:30:45.616670 12955 consensus_queue.cc:260] T 49f8c6571a09487cb520ff8dd8301cf9 P 9e5efa7a99fe42d287bb58c8fc0470ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "977bcfa24bb04ead85957eef52f199f5" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 36797 } } peers { permanent_uuid: "9e5efa7a99fe42d287bb58c8fc0470ea" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 38403 } } peers { permanent_uuid: "cea165d06fe944b68f3f05c704f508a8" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 38439 } }
I20260810 11:30:45.617008 12958 consensus_queue.cc:260] T 49f8c6571a09487cb520ff8dd8301cf9 P cea165d06fe944b68f3f05c704f508a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "977bcfa24bb04ead85957eef52f199f5" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 36797 } } peers { permanent_uuid: "9e5efa7a99fe42d287bb58c8fc0470ea" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 38403 } } peers { permanent_uuid: "cea165d06fe944b68f3f05c704f508a8" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 38439 } }
I20260810 11:30:45.619336 12958 ts_tablet_manager.cc:1428] T 49f8c6571a09487cb520ff8dd8301cf9 P cea165d06fe944b68f3f05c704f508a8: Time spent starting tablet: real 0.014s user 0.011s sys 0.004s
I20260810 11:30:45.621130 12799 heartbeater.cc:503] Master 127.8.18.254:39185 was elected leader, sending a full tablet report...
I20260810 11:30:45.621949 12955 ts_tablet_manager.cc:1428] T 49f8c6571a09487cb520ff8dd8301cf9 P 9e5efa7a99fe42d287bb58c8fc0470ea: Time spent starting tablet: real 0.021s user 0.015s sys 0.000s
W20260810 11:30:45.667622 12800 tablet.cc:2369] T 49f8c6571a09487cb520ff8dd8301cf9 P 9e5efa7a99fe42d287bb58c8fc0470ea: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 11:30:45.720232 12936 tablet.cc:2369] T 49f8c6571a09487cb520ff8dd8301cf9 P 977bcfa24bb04ead85957eef52f199f5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 11:30:45.751574 12964 raft_consensus.cc:482] T 49f8c6571a09487cb520ff8dd8301cf9 P cea165d06fe944b68f3f05c704f508a8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 11:30:45.751991 12964 raft_consensus.cc:504] T 49f8c6571a09487cb520ff8dd8301cf9 P cea165d06fe944b68f3f05c704f508a8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "977bcfa24bb04ead85957eef52f199f5" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 36797 } } peers { permanent_uuid: "9e5efa7a99fe42d287bb58c8fc0470ea" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 38403 } } peers { permanent_uuid: "cea165d06fe944b68f3f05c704f508a8" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 38439 } }
I20260810 11:30:45.753952 12964 leader_election.cc:288] T 49f8c6571a09487cb520ff8dd8301cf9 P cea165d06fe944b68f3f05c704f508a8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 977bcfa24bb04ead85957eef52f199f5 (127.8.18.195:36797), 9e5efa7a99fe42d287bb58c8fc0470ea (127.8.18.194:38403)
I20260810 11:30:45.761703 12949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:45.754544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.193:47345 (local address 127.8.18.194:38403)
0810 11:30:45.754748 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0810 11:30:45.754755 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:30:45.755808 (+ 1053us) server_negotiation.cc:407] Connection header received
0810 11:30:45.756019 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:30:45.756023 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:30:45.756122 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:30:45.756222 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:30:45.756888 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:45.757723 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:45.759749 (+ 2026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:45.760561 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:45.760696 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:45.760804 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0810 11:30:45.761232 (+ 428us) server_negotiation.cc:299] Negotiation successful
0810 11:30:45.761384 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260810 11:30:45.761744 12950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:45.754380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.193:60465 (local address 127.8.18.195:36797)
0810 11:30:45.754647 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0810 11:30:45.754655 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:30:45.755717 (+ 1062us) server_negotiation.cc:407] Connection header received
0810 11:30:45.755942 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:30:45.755947 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:30:45.756033 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:30:45.756104 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:30:45.756927 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:45.757725 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:45.759800 (+ 2075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:45.760649 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:45.760838 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:45.760915 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0810 11:30:45.761311 (+ 396us) server_negotiation.cc:299] Negotiation successful
0810 11:30:45.761476 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260810 11:30:45.761888 12966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:45.754579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.194:38403 (local address 127.8.18.193:47345)
0810 11:30:45.755627 (+ 1048us) negotiation.cc:107] Waiting for socket to connect
0810 11:30:45.755643 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 11:30:45.755855 (+ 212us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:30:45.756283 (+ 428us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:30:45.756288 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:30:45.756333 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:30:45.756683 (+ 350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:30:45.756693 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:30:45.757883 (+ 1190us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:30:45.757890 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:30:45.759549 (+ 1659us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:30:45.759566 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:30:45.760826 (+ 1260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:30:45.760832 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:30:45.761051 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:45.761078 (+ 27us) client_negotiation.cc:769] Sending connection context
0810 11:30:45.761164 (+ 86us) client_negotiation.cc:240] Negotiation successful
0810 11:30:45.761347 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":913,"mutex_wait_us":27,"thread_start_us":611,"threads_started":1}
I20260810 11:30:45.761912 12965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:45.754565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.195:36797 (local address 127.8.18.193:60465)
0810 11:30:45.755545 (+ 980us) negotiation.cc:107] Waiting for socket to connect
0810 11:30:45.755569 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0810 11:30:45.755764 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:30:45.756155 (+ 391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:30:45.756162 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:30:45.756195 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:30:45.756722 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:30:45.756736 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:30:45.757896 (+ 1160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:30:45.757900 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:30:45.759566 (+ 1666us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:30:45.759579 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:30:45.760914 (+ 1335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:30:45.760922 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:30:45.761099 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:45.761125 (+ 26us) client_negotiation.cc:769] Sending connection context
0810 11:30:45.761249 (+ 124us) client_negotiation.cc:240] Negotiation successful
0810 11:30:45.761391 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":846,"mutex_wait_us":49,"thread_start_us":500,"threads_started":1}
I20260810 11:30:45.763284 12890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "49f8c6571a09487cb520ff8dd8301cf9" candidate_uuid: "cea165d06fe944b68f3f05c704f508a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "977bcfa24bb04ead85957eef52f199f5" is_pre_election: true
I20260810 11:30:45.763280 12754 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "49f8c6571a09487cb520ff8dd8301cf9" candidate_uuid: "cea165d06fe944b68f3f05c704f508a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e5efa7a99fe42d287bb58c8fc0470ea" is_pre_election: true
I20260810 11:30:45.763864 12890 raft_consensus.cc:2454] T 49f8c6571a09487cb520ff8dd8301cf9 P 977bcfa24bb04ead85957eef52f199f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cea165d06fe944b68f3f05c704f508a8 in term 0.
I20260810 11:30:45.763864 12754 raft_consensus.cc:2454] T 49f8c6571a09487cb520ff8dd8301cf9 P 9e5efa7a99fe42d287bb58c8fc0470ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cea165d06fe944b68f3f05c704f508a8 in term 0.
I20260810 11:30:45.764640 12551 leader_election.cc:302] T 49f8c6571a09487cb520ff8dd8301cf9 P cea165d06fe944b68f3f05c704f508a8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 977bcfa24bb04ead85957eef52f199f5, cea165d06fe944b68f3f05c704f508a8; no voters:
I20260810 11:30:45.765100 12964 raft_consensus.cc:2781] T 49f8c6571a09487cb520ff8dd8301cf9 P cea165d06fe944b68f3f05c704f508a8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 11:30:45.765266 12964 raft_consensus.cc:482] T 49f8c6571a09487cb520ff8dd8301cf9 P cea165d06fe944b68f3f05c704f508a8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 11:30:45.765380 12964 raft_consensus.cc:3037] T 49f8c6571a09487cb520ff8dd8301cf9 P cea165d06fe944b68f3f05c704f508a8 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:30:45.767804 12964 raft_consensus.cc:504] T 49f8c6571a09487cb520ff8dd8301cf9 P cea165d06fe944b68f3f05c704f508a8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "977bcfa24bb04ead85957eef52f199f5" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 36797 } } peers { permanent_uuid: "9e5efa7a99fe42d287bb58c8fc0470ea" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 38403 } } peers { permanent_uuid: "cea165d06fe944b68f3f05c704f508a8" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 38439 } }
I20260810 11:30:45.768540 12964 leader_election.cc:288] T 49f8c6571a09487cb520ff8dd8301cf9 P cea165d06fe944b68f3f05c704f508a8 [CANDIDATE]: Term 1 election: Requested vote from peers 977bcfa24bb04ead85957eef52f199f5 (127.8.18.195:36797), 9e5efa7a99fe42d287bb58c8fc0470ea (127.8.18.194:38403)
I20260810 11:30:45.769268 12890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "49f8c6571a09487cb520ff8dd8301cf9" candidate_uuid: "cea165d06fe944b68f3f05c704f508a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "977bcfa24bb04ead85957eef52f199f5"
I20260810 11:30:45.769620 12890 raft_consensus.cc:3037] T 49f8c6571a09487cb520ff8dd8301cf9 P 977bcfa24bb04ead85957eef52f199f5 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:30:45.769599 12754 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "49f8c6571a09487cb520ff8dd8301cf9" candidate_uuid: "cea165d06fe944b68f3f05c704f508a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e5efa7a99fe42d287bb58c8fc0470ea"
I20260810 11:30:45.769850 12754 raft_consensus.cc:3037] T 49f8c6571a09487cb520ff8dd8301cf9 P 9e5efa7a99fe42d287bb58c8fc0470ea [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:30:45.772197 12890 raft_consensus.cc:2454] T 49f8c6571a09487cb520ff8dd8301cf9 P 977bcfa24bb04ead85957eef52f199f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cea165d06fe944b68f3f05c704f508a8 in term 1.
I20260810 11:30:45.772578 12754 raft_consensus.cc:2454] T 49f8c6571a09487cb520ff8dd8301cf9 P 9e5efa7a99fe42d287bb58c8fc0470ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cea165d06fe944b68f3f05c704f508a8 in term 1.
I20260810 11:30:45.772923 12551 leader_election.cc:302] T 49f8c6571a09487cb520ff8dd8301cf9 P cea165d06fe944b68f3f05c704f508a8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 977bcfa24bb04ead85957eef52f199f5, cea165d06fe944b68f3f05c704f508a8; no voters:
I20260810 11:30:45.773350 12964 raft_consensus.cc:2781] T 49f8c6571a09487cb520ff8dd8301cf9 P cea165d06fe944b68f3f05c704f508a8 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:30:45.773947 12964 raft_consensus.cc:686] T 49f8c6571a09487cb520ff8dd8301cf9 P cea165d06fe944b68f3f05c704f508a8 [term 1 LEADER]: Becoming Leader. State: Replica: cea165d06fe944b68f3f05c704f508a8, State: Running, Role: LEADER
I20260810 11:30:45.774827 12964 consensus_queue.cc:237] T 49f8c6571a09487cb520ff8dd8301cf9 P cea165d06fe944b68f3f05c704f508a8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "977bcfa24bb04ead85957eef52f199f5" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 36797 } } peers { permanent_uuid: "9e5efa7a99fe42d287bb58c8fc0470ea" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 38403 } } peers { permanent_uuid: "cea165d06fe944b68f3f05c704f508a8" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 38439 } }
I20260810 11:30:45.780504 12468 catalog_manager.cc:5495] T 49f8c6571a09487cb520ff8dd8301cf9 P cea165d06fe944b68f3f05c704f508a8 reported cstate change: term changed from 0 to 1, leader changed from <none> to cea165d06fe944b68f3f05c704f508a8 (127.8.18.193). New cstate: current_term: 1 leader_uuid: "cea165d06fe944b68f3f05c704f508a8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "977bcfa24bb04ead85957eef52f199f5" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 36797 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9e5efa7a99fe42d287bb58c8fc0470ea" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 38403 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cea165d06fe944b68f3f05c704f508a8" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 38439 } health_report { overall_health: HEALTHY } } }
I20260810 11:30:45.812441 12954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:45.805840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37282 (local address 127.8.18.193:38439)
0810 11:30:45.806089 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0810 11:30:45.806095 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:30:45.806247 (+ 152us) server_negotiation.cc:407] Connection header received
0810 11:30:45.806525 (+ 278us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:30:45.806529 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:30:45.806631 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:30:45.806792 (+ 161us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 11:30:45.807333 (+ 541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:45.808064 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:45.809590 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:45.810127 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:45.810257 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:45.810935 (+ 678us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 11:30:45.811647 (+ 712us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 11:30:45.811774 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0810 11:30:45.812095 (+ 321us) server_negotiation.cc:299] Negotiation successful
0810 11:30:45.812198 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260810 11:30:45.821768 12754 raft_consensus.cc:1261] T 49f8c6571a09487cb520ff8dd8301cf9 P 9e5efa7a99fe42d287bb58c8fc0470ea [term 1 FOLLOWER]: Refusing update from remote peer cea165d06fe944b68f3f05c704f508a8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 11:30:45.821826 12890 raft_consensus.cc:1261] T 49f8c6571a09487cb520ff8dd8301cf9 P 977bcfa24bb04ead85957eef52f199f5 [term 1 FOLLOWER]: Refusing update from remote peer cea165d06fe944b68f3f05c704f508a8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 11:30:45.823124 12967 consensus_queue.cc:1035] T 49f8c6571a09487cb520ff8dd8301cf9 P cea165d06fe944b68f3f05c704f508a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e5efa7a99fe42d287bb58c8fc0470ea" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 38403 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 11:30:45.823587 12964 consensus_queue.cc:1035] T 49f8c6571a09487cb520ff8dd8301cf9 P cea165d06fe944b68f3f05c704f508a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "977bcfa24bb04ead85957eef52f199f5" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 36797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 11:30:45.837111 12970 mvcc.cc:205] Tried to move back new op lower bound from 7316936482074406912 to 7316936481898123264. Current Snapshot: MvccSnapshot[applied={T|T < 7316936482074406912}]
I20260810 11:30:45.839625 12971 mvcc.cc:205] Tried to move back new op lower bound from 7316936482074406912 to 7316936481898123264. Current Snapshot: MvccSnapshot[applied={T|T < 7316936482074406912}]
I20260810 11:30:45.855346 12469 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48062:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 11:30:45.855688 12469 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48062:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 11:30:45.860132 12469 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4fbdd77c18624f98b24cf1c7175929ad: Sending DeleteTablet for 3 replicas of tablet 49f8c6571a09487cb520ff8dd8301cf9
I20260810 11:30:45.861768 12733 tablet_service.cc:1514] Processing DeleteTablet for tablet 49f8c6571a09487cb520ff8dd8301cf9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:30:45 UTC) from {username='kudu', principal='kudu/127.8.18.254@KRBTEST.COM'} at 127.0.0.1:45670
I20260810 11:30:45.861756 12870 tablet_service.cc:1514] Processing DeleteTablet for tablet 49f8c6571a09487cb520ff8dd8301cf9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:30:45 UTC) from {username='kudu', principal='kudu/127.8.18.254@KRBTEST.COM'} at 127.0.0.1:41920
I20260810 11:30:45.862012 12598 tablet_service.cc:1514] Processing DeleteTablet for tablet 49f8c6571a09487cb520ff8dd8301cf9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:30:45 UTC) from {username='kudu', principal='kudu/127.8.18.254@KRBTEST.COM'} at 127.0.0.1:37276
I20260810 11:30:45.867998 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 12533
I20260810 11:30:45.874401 12982 tablet_replica.cc:330] T 49f8c6571a09487cb520ff8dd8301cf9 P cea165d06fe944b68f3f05c704f508a8: stopping tablet replica
I20260810 11:30:45.875031 12982 raft_consensus.cc:2229] T 49f8c6571a09487cb520ff8dd8301cf9 P cea165d06fe944b68f3f05c704f508a8 [term 1 LEADER]: Raft consensus shutting down.
I20260810 11:30:45.875824 12982 raft_consensus.cc:2258] T 49f8c6571a09487cb520ff8dd8301cf9 P cea165d06fe944b68f3f05c704f508a8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:30:45.876649 12981 tablet_replica.cc:330] T 49f8c6571a09487cb520ff8dd8301cf9 P 977bcfa24bb04ead85957eef52f199f5: stopping tablet replica
I20260810 11:30:45.877437 12981 raft_consensus.cc:2229] T 49f8c6571a09487cb520ff8dd8301cf9 P 977bcfa24bb04ead85957eef52f199f5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 11:30:45.878186 12981 raft_consensus.cc:2258] T 49f8c6571a09487cb520ff8dd8301cf9 P 977bcfa24bb04ead85957eef52f199f5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:30:45.878634 12982 ts_tablet_manager.cc:1905] T 49f8c6571a09487cb520ff8dd8301cf9 P cea165d06fe944b68f3f05c704f508a8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:30:45.881068 12981 ts_tablet_manager.cc:1905] T 49f8c6571a09487cb520ff8dd8301cf9 P 977bcfa24bb04ead85957eef52f199f5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:30:45.887866 12982 ts_tablet_manager.cc:1918] T 49f8c6571a09487cb520ff8dd8301cf9 P cea165d06fe944b68f3f05c704f508a8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 11:30:45.888098 12982 log.cc:1196] T 49f8c6571a09487cb520ff8dd8301cf9 P cea165d06fe944b68f3f05c704f508a8: Deleting WAL directory at /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786361401922362-8267-0/minicluster-data/ts-0/wal/wals/49f8c6571a09487cb520ff8dd8301cf9
I20260810 11:30:45.888782 12982 ts_tablet_manager.cc:1939] T 49f8c6571a09487cb520ff8dd8301cf9 P cea165d06fe944b68f3f05c704f508a8: Deleting consensus metadata
I20260810 11:30:45.891067 12457 catalog_manager.cc:4841] TS cea165d06fe944b68f3f05c704f508a8 (127.8.18.193:38439): tablet 49f8c6571a09487cb520ff8dd8301cf9 (table test-table [id=c68eb4f1ef7140928ce896fc53ae2d90]) successfully deleted
I20260810 11:30:45.891801 12981 ts_tablet_manager.cc:1918] T 49f8c6571a09487cb520ff8dd8301cf9 P 977bcfa24bb04ead85957eef52f199f5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 11:30:45.892087 12980 tablet_replica.cc:330] T 49f8c6571a09487cb520ff8dd8301cf9 P 9e5efa7a99fe42d287bb58c8fc0470ea: stopping tablet replica
I20260810 11:30:45.892434 12981 log.cc:1196] T 49f8c6571a09487cb520ff8dd8301cf9 P 977bcfa24bb04ead85957eef52f199f5: Deleting WAL directory at /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786361401922362-8267-0/minicluster-data/ts-2/wal/wals/49f8c6571a09487cb520ff8dd8301cf9
I20260810 11:30:45.892639 12980 raft_consensus.cc:2229] T 49f8c6571a09487cb520ff8dd8301cf9 P 9e5efa7a99fe42d287bb58c8fc0470ea [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 11:30:45.893281 12981 ts_tablet_manager.cc:1939] T 49f8c6571a09487cb520ff8dd8301cf9 P 977bcfa24bb04ead85957eef52f199f5: Deleting consensus metadata
I20260810 11:30:45.893261 12980 raft_consensus.cc:2258] T 49f8c6571a09487cb520ff8dd8301cf9 P 9e5efa7a99fe42d287bb58c8fc0470ea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:30:45.895532 12457 catalog_manager.cc:4841] TS 977bcfa24bb04ead85957eef52f199f5 (127.8.18.195:36797): tablet 49f8c6571a09487cb520ff8dd8301cf9 (table test-table [id=c68eb4f1ef7140928ce896fc53ae2d90]) successfully deleted
I20260810 11:30:45.895905 12980 ts_tablet_manager.cc:1905] T 49f8c6571a09487cb520ff8dd8301cf9 P 9e5efa7a99fe42d287bb58c8fc0470ea: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:30:45.903977 12980 ts_tablet_manager.cc:1918] T 49f8c6571a09487cb520ff8dd8301cf9 P 9e5efa7a99fe42d287bb58c8fc0470ea: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 11:30:45.904250 12980 log.cc:1196] T 49f8c6571a09487cb520ff8dd8301cf9 P 9e5efa7a99fe42d287bb58c8fc0470ea: Deleting WAL directory at /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786361401922362-8267-0/minicluster-data/ts-1/wal/wals/49f8c6571a09487cb520ff8dd8301cf9
I20260810 11:30:45.904882 12980 ts_tablet_manager.cc:1939] T 49f8c6571a09487cb520ff8dd8301cf9 P 9e5efa7a99fe42d287bb58c8fc0470ea: Deleting consensus metadata
Aug 10 11:30:45 dist-test-slave-srgv krb5kdc[12425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361443, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.18.193@KRBTEST.COM
I20260810 11:30:45.907162 12456 catalog_manager.cc:4841] TS 9e5efa7a99fe42d287bb58c8fc0470ea (127.8.18.194:38403): tablet 49f8c6571a09487cb520ff8dd8301cf9 (table test-table [id=c68eb4f1ef7140928ce896fc53ae2d90]) successfully deleted
I20260810 11:30:45.916851 12954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:45.869606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37288 (local address 127.8.18.193:38439)
0810 11:30:45.869871 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0810 11:30:45.869881 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 11:30:45.892082 (+ 22201us) server_negotiation.cc:407] Connection header received
0810 11:30:45.892608 (+ 526us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:30:45.892616 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:30:45.892721 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:30:45.892833 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:30:45.895227 (+ 2394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:45.896395 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:45.897834 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:45.898324 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:45.898496 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:45.907993 (+ 9497us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:30:45.908015 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:30:45.908036 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:30:45.908098 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:30:45.912829 (+ 4731us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:45.913956 (+ 1127us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:45.913966 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:45.913969 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:45.914188 (+ 219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:45.914736 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:45.914742 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:45.914748 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:45.915229 (+ 481us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:30:45.915565 (+ 336us) server_negotiation.cc:1035] Waiting for connection context
0810 11:30:45.915948 (+ 383us) server_negotiation.cc:299] Negotiation successful
0810 11:30:45.916218 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260810 11:30:45.917708 12658 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:30:46.029117 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 12533
I20260810 11:30:46.049579 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 12669
Aug 10 11:30:46 dist-test-slave-srgv krb5kdc[12425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361443, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.18.194@KRBTEST.COM
I20260810 11:30:46.066890 12949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:46.050421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45682 (local address 127.8.18.194:38403)
0810 11:30:46.050629 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0810 11:30:46.050635 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:30:46.050658 (+ 23us) server_negotiation.cc:407] Connection header received
0810 11:30:46.050825 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:30:46.050829 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:30:46.050908 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:30:46.050999 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:30:46.052394 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:46.053172 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:46.054690 (+ 1518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:46.055179 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:46.055302 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:46.060202 (+ 4900us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:30:46.060221 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:30:46.060248 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:30:46.060292 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:30:46.063770 (+ 3478us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:46.064785 (+ 1015us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:46.064794 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:46.064797 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:46.064947 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:46.065371 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:46.065374 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:46.065376 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:46.065839 (+ 463us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:30:46.066077 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0810 11:30:46.066422 (+ 345us) server_negotiation.cc:299] Negotiation successful
0810 11:30:46.066617 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260810 11:30:46.067664 12794 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:30:46.159482 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 12669
I20260810 11:30:46.178436 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 12805
Aug 10 11:30:46 dist-test-slave-srgv krb5kdc[12425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361443, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.18.195@KRBTEST.COM
I20260810 11:30:46.198715 12950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:46.179333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41922 (local address 127.8.18.195:36797)
0810 11:30:46.179509 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0810 11:30:46.179513 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 11:30:46.179534 (+ 21us) server_negotiation.cc:407] Connection header received
0810 11:30:46.179593 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:30:46.179597 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:30:46.179662 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:30:46.179759 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:30:46.181840 (+ 2081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:46.183173 (+ 1333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:46.185638 (+ 2465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:46.186360 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:46.186525 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:46.192499 (+ 5974us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:30:46.192517 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:30:46.192542 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:30:46.192599 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:30:46.195749 (+ 3150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:46.196546 (+ 797us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:46.196559 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:46.196562 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:46.196739 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:46.197107 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:46.197111 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:46.197113 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:46.197591 (+ 478us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:30:46.197874 (+ 283us) server_negotiation.cc:1035] Waiting for connection context
0810 11:30:46.198123 (+ 249us) server_negotiation.cc:299] Negotiation successful
0810 11:30:46.198320 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260810 11:30:46.199412 12930 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:30:46.293166 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 12805
I20260810 11:30:46.311532 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 12438
I20260810 11:30:46.312857 12501 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:30:46.427073 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 12438
2026-08-10T11:30:46Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (4553 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-10T11:30:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T11:30:46Z Disabled control of system clock
I20260810 11:30:46.505944 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786361401922362-8267-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786361401922362-8267-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786361401922362-8267-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.18.254:34401
--webserver_interface=127.8.18.254
--webserver_port=0
--builtin_ntp_servers=127.8.18.212:45671
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.18.254:34401
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260810 11:30:46.877883 12991 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:30:46.878268 12991 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:30:46.878489 12991 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:30:46.886612 12991 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 11:30:46.886767 12991 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:30:46.886858 12991 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:30:46.886950 12991 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 11:30:46.887030 12991 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 11:30:46.897205 12991 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.18.212:45671
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786361401922362-8267-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786361401922362-8267-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.18.254:34401
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.18.254:34401
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.18.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786361401922362-8267-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:30:46.898736 12991 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:30:46.900844 12991 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:30:46.914148 12998 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:30:46.914222 12999 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:30:46.914186 13001 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:30:46.915244 12991 server_base.cc:1034] running on GCE node
I20260810 11:30:46.916198 12991 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:30:46.917939 12991 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:30:46.919288 12991 hybrid_clock.cc:648] HybridClock initialized: now 1786361446919240 us; error 68 us; skew 500 ppm
I20260810 11:30:46.919721 12991 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 11:30:46.922802 12991 webserver.cc:460] Webserver started at http://127.8.18.254:44437/ using document root <none> and password file <none>
I20260810 11:30:46.923640 12991 fs_manager.cc:359] Metadata directory not provided
I20260810 11:30:46.923799 12991 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:30:46.924193 12991 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:30:46.927603 12991 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786361401922362-8267-0/minicluster-data/master-0/data/instance:
uuid: "324304b5aacc45239b49bb96dae05f6c"
format_stamp: "Formatted at 2026-08-10 11:30:46 on dist-test-slave-srgv"
server_key: "3ecbd00eadf99c79289b0cc9881d2d2a"
server_key_iv: "f254adfa23c159aafdcac2efca11a4d4"
server_key_version: "encryptionkey@0"
I20260810 11:30:46.928682 12991 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786361401922362-8267-0/minicluster-data/master-0/wal/instance:
uuid: "324304b5aacc45239b49bb96dae05f6c"
format_stamp: "Formatted at 2026-08-10 11:30:46 on dist-test-slave-srgv"
server_key: "3ecbd00eadf99c79289b0cc9881d2d2a"
server_key_iv: "f254adfa23c159aafdcac2efca11a4d4"
server_key_version: "encryptionkey@0"
I20260810 11:30:46.935105 12991 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.000s sys 0.008s
I20260810 11:30:46.939729 13007 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:30:46.941490 12991 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260810 11:30:46.941758 12991 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786361401922362-8267-0/minicluster-data/master-0/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786361401922362-8267-0/minicluster-data/master-0/wal
uuid: "324304b5aacc45239b49bb96dae05f6c"
format_stamp: "Formatted at 2026-08-10 11:30:46 on dist-test-slave-srgv"
server_key: "3ecbd00eadf99c79289b0cc9881d2d2a"
server_key_iv: "f254adfa23c159aafdcac2efca11a4d4"
server_key_version: "encryptionkey@0"
I20260810 11:30:46.942054 12991 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786361401922362-8267-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786361401922362-8267-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786361401922362-8267-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:30:46.948726 12991 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:30:46.949808 12991 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:30:46.950186 12991 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:30:46.971318 12991 rpc_server.cc:307] RPC server started. Bound to: 127.8.18.254:34401
I20260810 11:30:46.971386 13059 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.18.254:34401 every 8 connection(s)
I20260810 11:30:46.972923 12991 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
I20260810 11:30:46.977298 13060 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:30:46.981868 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 12991
I20260810 11:30:46.982124 8267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786361401922362-8267-0/minicluster-data/master-0/wal/instance
I20260810 11:30:46.982842 8267 external_mini_cluster.cc:1428] Setting key 14e1fa2487d3b65302b126e3a2370700
I20260810 11:30:46.991804 13060 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 324304b5aacc45239b49bb96dae05f6c: Bootstrap starting.
I20260810 11:30:46.994654 13063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:46.985489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44020 (local address 127.8.18.254:34401)
0810 11:30:46.986859 (+ 1370us) server_negotiation.cc:206] Beginning negotiation
0810 11:30:46.986878 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0810 11:30:46.986945 (+ 67us) server_negotiation.cc:407] Connection header received
0810 11:30:46.989696 (+ 2751us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:30:46.989736 (+ 40us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:30:46.990432 (+ 696us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:30:46.990971 (+ 539us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:30:46.992337 (+ 1366us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:30:46.992346 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:30:46.992366 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:30:46.992386 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:30:46.992483 (+ 97us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:30:46.992511 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:30:46.992758 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0810 11:30:46.992907 (+ 149us) server_negotiation.cc:299] Negotiation successful
0810 11:30:46.993202 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":678,"thread_start_us":409,"threads_started":1}
I20260810 11:30:46.996405 13060 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 324304b5aacc45239b49bb96dae05f6c: Neither blocks nor log segments found. Creating new log.
I20260810 11:30:46.997737 13060 log.cc:824] T 00000000000000000000000000000000 P 324304b5aacc45239b49bb96dae05f6c: Log is configured to *not* fsync() on all Append() calls
I20260810 11:30:47.001104 13060 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 324304b5aacc45239b49bb96dae05f6c: No bootstrap required, opened a new log
I20260810 11:30:47.009766 13060 raft_consensus.cc:348] T 00000000000000000000000000000000 P 324304b5aacc45239b49bb96dae05f6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "324304b5aacc45239b49bb96dae05f6c" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 34401 } }
I20260810 11:30:47.010241 13060 raft_consensus.cc:374] T 00000000000000000000000000000000 P 324304b5aacc45239b49bb96dae05f6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:30:47.010368 13060 raft_consensus.cc:729] T 00000000000000000000000000000000 P 324304b5aacc45239b49bb96dae05f6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 324304b5aacc45239b49bb96dae05f6c, State: Initialized, Role: FOLLOWER
I20260810 11:30:47.011075 13060 consensus_queue.cc:260] T 00000000000000000000000000000000 P 324304b5aacc45239b49bb96dae05f6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "324304b5aacc45239b49bb96dae05f6c" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 34401 } }
I20260810 11:30:47.011359 13060 raft_consensus.cc:388] T 00000000000000000000000000000000 P 324304b5aacc45239b49bb96dae05f6c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 11:30:47.011518 13060 raft_consensus.cc:482] T 00000000000000000000000000000000 P 324304b5aacc45239b49bb96dae05f6c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 11:30:47.011708 13060 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 324304b5aacc45239b49bb96dae05f6c [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:30:47.014424 13060 raft_consensus.cc:504] T 00000000000000000000000000000000 P 324304b5aacc45239b49bb96dae05f6c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "324304b5aacc45239b49bb96dae05f6c" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 34401 } }
I20260810 11:30:47.015054 13060 leader_election.cc:302] T 00000000000000000000000000000000 P 324304b5aacc45239b49bb96dae05f6c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 324304b5aacc45239b49bb96dae05f6c; no voters:
I20260810 11:30:47.015813 13060 leader_election.cc:288] T 00000000000000000000000000000000 P 324304b5aacc45239b49bb96dae05f6c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 11:30:47.015942 13065 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 324304b5aacc45239b49bb96dae05f6c [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:30:47.016896 13065 raft_consensus.cc:686] T 00000000000000000000000000000000 P 324304b5aacc45239b49bb96dae05f6c [term 1 LEADER]: Becoming Leader. State: Replica: 324304b5aacc45239b49bb96dae05f6c, State: Running, Role: LEADER
I20260810 11:30:47.017627 13065 consensus_queue.cc:237] T 00000000000000000000000000000000 P 324304b5aacc45239b49bb96dae05f6c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "324304b5aacc45239b49bb96dae05f6c" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 34401 } }
I20260810 11:30:47.018553 13060 sys_catalog.cc:564] T 00000000000000000000000000000000 P 324304b5aacc45239b49bb96dae05f6c [sys.catalog]: configured and running, proceeding with master startup.
I20260810 11:30:47.021200 13067 sys_catalog.cc:455] T 00000000000000000000000000000000 P 324304b5aacc45239b49bb96dae05f6c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 324304b5aacc45239b49bb96dae05f6c. Latest consensus state: current_term: 1 leader_uuid: "324304b5aacc45239b49bb96dae05f6c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "324304b5aacc45239b49bb96dae05f6c" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 34401 } } }
I20260810 11:30:47.021857 13067 sys_catalog.cc:458] T 00000000000000000000000000000000 P 324304b5aacc45239b49bb96dae05f6c [sys.catalog]: This master's current role is: LEADER
I20260810 11:30:47.021238 13066 sys_catalog.cc:455] T 00000000000000000000000000000000 P 324304b5aacc45239b49bb96dae05f6c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "324304b5aacc45239b49bb96dae05f6c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "324304b5aacc45239b49bb96dae05f6c" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 34401 } } }
I20260810 11:30:47.023036 13066 sys_catalog.cc:458] T 00000000000000000000000000000000 P 324304b5aacc45239b49bb96dae05f6c [sys.catalog]: This master's current role is: LEADER
I20260810 11:30:47.029784 13073 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 11:30:47.036106 13073 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 11:30:47.047502 13073 catalog_manager.cc:1347] Generated new cluster ID: ca45226fdea0487ea46b2c0a4d5ca3e9
I20260810 11:30:47.047664 13073 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 11:30:47.069763 13073 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 11:30:47.070379 13073 catalog_manager.cc:1504] Loading token signing keys...
I20260810 11:30:47.079679 13073 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 324304b5aacc45239b49bb96dae05f6c: Generated new TSK 0
I20260810 11:30:47.080997 13073 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 11:30:47.099084 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786361401922362-8267-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786361401922362-8267-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786361401922362-8267-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786361401922362-8267-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.18.193:0
--local_ip_for_outbound_sockets=127.8.18.193
--webserver_interface=127.8.18.193
--webserver_port=0
--tserver_master_addrs=127.8.18.254:34401
--builtin_ntp_servers=127.8.18.212:45671
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260810 11:30:47.473888 13085 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:30:47.474267 13085 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:30:47.474450 13085 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:30:47.482558 13085 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:30:47.482759 13085 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:30:47.482954 13085 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.18.193
I20260810 11:30:47.493494 13085 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.18.212:45671
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786361401922362-8267-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786361401922362-8267-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.18.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786361401922362-8267-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.18.193
--webserver_port=0
--tserver_master_addrs=127.8.18.254:34401
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.18.193
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786361401922362-8267-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:30:47.495102 13085 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:30:47.497186 13085 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:30:47.512504 13090 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:30:47.512326 13093 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:30:47.512352 13091 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:30:47.513362 13085 server_base.cc:1034] running on GCE node
I20260810 11:30:47.514194 13085 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:30:47.515337 13085 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:30:47.516614 13085 hybrid_clock.cc:648] HybridClock initialized: now 1786361447516555 us; error 55 us; skew 500 ppm
I20260810 11:30:47.516980 13085 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 11:30:47.519783 13085 webserver.cc:460] Webserver started at http://127.8.18.193:36663/ using document root <none> and password file <none>
I20260810 11:30:47.520617 13085 fs_manager.cc:359] Metadata directory not provided
I20260810 11:30:47.520766 13085 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:30:47.521131 13085 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:30:47.524698 13085 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786361401922362-8267-0/minicluster-data/ts-0/data/instance:
uuid: "31228a8ad6f6423f80644ea244c6fa24"
format_stamp: "Formatted at 2026-08-10 11:30:47 on dist-test-slave-srgv"
server_key: "f5c62a95cd0fe0b68040bbf3a7e13e1d"
server_key_iv: "268c6fcac81185d0da4c58878d0f0933"
server_key_version: "encryptionkey@0"
I20260810 11:30:47.525803 13085 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786361401922362-8267-0/minicluster-data/ts-0/wal/instance:
uuid: "31228a8ad6f6423f80644ea244c6fa24"
format_stamp: "Formatted at 2026-08-10 11:30:47 on dist-test-slave-srgv"
server_key: "f5c62a95cd0fe0b68040bbf3a7e13e1d"
server_key_iv: "268c6fcac81185d0da4c58878d0f0933"
server_key_version: "encryptionkey@0"
I20260810 11:30:47.532285 13085 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260810 11:30:47.536695 13100 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:30:47.538492 13085 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260810 11:30:47.538720 13085 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786361401922362-8267-0/minicluster-data/ts-0/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786361401922362-8267-0/minicluster-data/ts-0/wal
uuid: "31228a8ad6f6423f80644ea244c6fa24"
format_stamp: "Formatted at 2026-08-10 11:30:47 on dist-test-slave-srgv"
server_key: "f5c62a95cd0fe0b68040bbf3a7e13e1d"
server_key_iv: "268c6fcac81185d0da4c58878d0f0933"
server_key_version: "encryptionkey@0"
I20260810 11:30:47.538996 13085 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786361401922362-8267-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786361401922362-8267-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786361401922362-8267-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:30:47.545804 13085 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:30:47.546964 13085 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:30:47.547291 13085 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:30:47.548539 13085 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:30:47.550272 13085 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:30:47.550369 13085 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:30:47.550540 13085 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:30:47.550638 13085 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:30:47.587009 13085 rpc_server.cc:307] RPC server started. Bound to: 127.8.18.193:35711
I20260810 11:30:47.587076 13216 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.18.193:35711 every 8 connection(s)
I20260810 11:30:47.588486 13085 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786361401922362-8267-0/minicluster-data/ts-0/data/info.pb
I20260810 11:30:47.596680 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 13085
I20260810 11:30:47.596912 8267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786361401922362-8267-0/minicluster-data/ts-0/wal/instance
I20260810 11:30:47.597653 8267 external_mini_cluster.cc:1428] Setting key dfec00bfe725ca9caa6a91d98dcb1437
I20260810 11:30:47.597957 13219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:47.591437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.193:35205 (local address 127.8.18.254:34401)
0810 11:30:47.591929 (+ 492us) server_negotiation.cc:206] Beginning negotiation
0810 11:30:47.591936 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:30:47.593305 (+ 1369us) server_negotiation.cc:407] Connection header received
0810 11:30:47.595185 (+ 1880us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:30:47.595192 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:30:47.595250 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:30:47.595362 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:30:47.596950 (+ 1588us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:30:47.596955 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:30:47.596979 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:30:47.597006 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:30:47.597067 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:30:47.597082 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:30:47.597230 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0810 11:30:47.597534 (+ 304us) server_negotiation.cc:299] Negotiation successful
0810 11:30:47.597612 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":361,"thread_start_us":234,"threads_started":1}
I20260810 11:30:47.599138 13220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:47.592097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:34401 (local address 127.8.18.193:35205)
0810 11:30:47.593119 (+ 1022us) negotiation.cc:107] Waiting for socket to connect
0810 11:30:47.593161 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 11:30:47.594823 (+ 1662us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:30:47.595599 (+ 776us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:30:47.595607 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:30:47.595708 (+ 101us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:30:47.596527 (+ 819us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 11:30:47.596530 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:30:47.596572 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 11:30:47.596667 (+ 95us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:30:47.597191 (+ 524us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:30:47.597205 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:30:47.597213 (+ 8us) client_negotiation.cc:769] Sending connection context
0810 11:30:47.597639 (+ 426us) client_negotiation.cc:240] Negotiation successful
0810 11:30:47.597943 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":519,"thread_start_us":330,"threads_started":1}
I20260810 11:30:47.600883 13217 heartbeater.cc:347] Connected to a master server at 127.8.18.254:34401
I20260810 11:30:47.600929 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786361401922362-8267-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786361401922362-8267-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786361401922362-8267-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786361401922362-8267-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.18.194:0
--local_ip_for_outbound_sockets=127.8.18.194
--webserver_interface=127.8.18.194
--webserver_port=0
--tserver_master_addrs=127.8.18.254:34401
--builtin_ntp_servers=127.8.18.212:45671
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260810 11:30:47.601281 13217 heartbeater.cc:464] Registering TS with master...
I20260810 11:30:47.601985 13217 heartbeater.cc:511] Master 127.8.18.254:34401 requested a full tablet report, sending...
I20260810 11:30:47.604310 13024 ts_manager.cc:194] Registered new tserver with Master: 31228a8ad6f6423f80644ea244c6fa24 (127.8.18.193:35711)
W20260810 11:30:47.959833 13221 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:30:47.960224 13221 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:30:47.960409 13221 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:30:47.969036 13221 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:30:47.969218 13221 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:30:47.969388 13221 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.18.194
I20260810 11:30:47.979450 13221 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.18.212:45671
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786361401922362-8267-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786361401922362-8267-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.18.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786361401922362-8267-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.18.194
--webserver_port=0
--tserver_master_addrs=127.8.18.254:34401
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.18.194
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786361401922362-8267-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:30:47.980832 13221 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:30:47.982851 13221 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:30:47.997731 13226 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:30:47.997824 13229 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:30:47.997790 13227 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:30:47.998792 13221 server_base.cc:1034] running on GCE node
I20260810 11:30:47.999543 13221 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:30:48.000672 13221 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:30:48.001910 13221 hybrid_clock.cc:648] HybridClock initialized: now 1786361448001861 us; error 38 us; skew 500 ppm
I20260810 11:30:48.002323 13221 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 11:30:48.005035 13221 webserver.cc:460] Webserver started at http://127.8.18.194:44561/ using document root <none> and password file <none>
I20260810 11:30:48.005877 13221 fs_manager.cc:359] Metadata directory not provided
I20260810 11:30:48.006001 13221 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:30:48.006368 13221 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:30:48.009620 13221 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786361401922362-8267-0/minicluster-data/ts-1/data/instance:
uuid: "de9c26202bc14312b1c84312997acc41"
format_stamp: "Formatted at 2026-08-10 11:30:48 on dist-test-slave-srgv"
server_key: "b0ef36a9d0cf08f81442b780f52b84f6"
server_key_iv: "6ba573966d97181b43eeab5203d75dec"
server_key_version: "encryptionkey@0"
I20260810 11:30:48.010517 13221 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786361401922362-8267-0/minicluster-data/ts-1/wal/instance:
uuid: "de9c26202bc14312b1c84312997acc41"
format_stamp: "Formatted at 2026-08-10 11:30:48 on dist-test-slave-srgv"
server_key: "b0ef36a9d0cf08f81442b780f52b84f6"
server_key_iv: "6ba573966d97181b43eeab5203d75dec"
server_key_version: "encryptionkey@0"
I20260810 11:30:48.016566 13221 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260810 11:30:48.021049 13236 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:30:48.022895 13221 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260810 11:30:48.023139 13221 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786361401922362-8267-0/minicluster-data/ts-1/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786361401922362-8267-0/minicluster-data/ts-1/wal
uuid: "de9c26202bc14312b1c84312997acc41"
format_stamp: "Formatted at 2026-08-10 11:30:48 on dist-test-slave-srgv"
server_key: "b0ef36a9d0cf08f81442b780f52b84f6"
server_key_iv: "6ba573966d97181b43eeab5203d75dec"
server_key_version: "encryptionkey@0"
I20260810 11:30:48.023422 13221 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786361401922362-8267-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786361401922362-8267-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786361401922362-8267-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:30:48.030491 13221 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:30:48.031505 13221 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:30:48.031831 13221 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:30:48.032897 13221 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:30:48.034662 13221 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:30:48.034813 13221 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:30:48.034955 13221 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:30:48.035034 13221 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:30:48.072536 13221 rpc_server.cc:307] RPC server started. Bound to: 127.8.18.194:43759
I20260810 11:30:48.072551 13353 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.18.194:43759 every 8 connection(s)
I20260810 11:30:48.074085 13221 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786361401922362-8267-0/minicluster-data/ts-1/data/info.pb
I20260810 11:30:48.075847 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 13221
I20260810 11:30:48.076061 8267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786361401922362-8267-0/minicluster-data/ts-1/wal/instance
I20260810 11:30:48.076681 8267 external_mini_cluster.cc:1428] Setting key 9ac51c83fae522d23e689daadf01aedc
I20260810 11:30:48.080425 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786361401922362-8267-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786361401922362-8267-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786361401922362-8267-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786361401922362-8267-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.18.195:0
--local_ip_for_outbound_sockets=127.8.18.195
--webserver_interface=127.8.18.195
--webserver_port=0
--tserver_master_addrs=127.8.18.254:34401
--builtin_ntp_servers=127.8.18.212:45671
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260810 11:30:48.084985 13219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:48.077124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.194:44819 (local address 127.8.18.254:34401)
0810 11:30:48.077361 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0810 11:30:48.077370 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 11:30:48.079212 (+ 1842us) server_negotiation.cc:407] Connection header received
0810 11:30:48.082237 (+ 3025us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:30:48.082242 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:30:48.082301 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:30:48.082404 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:30:48.084223 (+ 1819us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:30:48.084228 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:30:48.084231 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:30:48.084251 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:30:48.084288 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:30:48.084300 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:30:48.084446 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0810 11:30:48.084679 (+ 233us) server_negotiation.cc:299] Negotiation successful
0810 11:30:48.084749 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260810 11:30:48.086582 13356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:48.077862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:34401 (local address 127.8.18.194:44819)
0810 11:30:48.079052 (+ 1190us) negotiation.cc:107] Waiting for socket to connect
0810 11:30:48.079094 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 11:30:48.081832 (+ 2738us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:30:48.082758 (+ 926us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:30:48.082777 (+ 19us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:30:48.082934 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:30:48.083748 (+ 814us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 11:30:48.083754 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:30:48.083813 (+ 59us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 11:30:48.083958 (+ 145us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:30:48.084431 (+ 473us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:30:48.084444 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:30:48.084454 (+ 10us) client_negotiation.cc:769] Sending connection context
0810 11:30:48.084789 (+ 335us) client_negotiation.cc:240] Negotiation successful
0810 11:30:48.085133 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":608,"thread_start_us":363,"threads_started":1}
I20260810 11:30:48.088389 13354 heartbeater.cc:347] Connected to a master server at 127.8.18.254:34401
I20260810 11:30:48.088802 13354 heartbeater.cc:464] Registering TS with master...
I20260810 11:30:48.089720 13354 heartbeater.cc:511] Master 127.8.18.254:34401 requested a full tablet report, sending...
I20260810 11:30:48.091650 13024 ts_manager.cc:194] Registered new tserver with Master: de9c26202bc14312b1c84312997acc41 (127.8.18.194:43759)
W20260810 11:30:48.445106 13357 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:30:48.445550 13357 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:30:48.445734 13357 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:30:48.453771 13357 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:30:48.453904 13357 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:30:48.454113 13357 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.18.195
I20260810 11:30:48.463891 13357 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.18.212:45671
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786361401922362-8267-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786361401922362-8267-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.18.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786361401922362-8267-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.18.195
--webserver_port=0
--tserver_master_addrs=127.8.18.254:34401
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.18.195
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786361401922362-8267-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:30:48.465371 13357 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:30:48.467367 13357 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:30:48.482125 13363 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:30:48.482231 13365 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:30:48.482189 13362 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:30:48.483604 13357 server_base.cc:1034] running on GCE node
I20260810 11:30:48.484274 13357 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:30:48.485308 13357 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:30:48.486524 13357 hybrid_clock.cc:648] HybridClock initialized: now 1786361448486480 us; error 36 us; skew 500 ppm
I20260810 11:30:48.486848 13357 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 11:30:48.489382 13357 webserver.cc:460] Webserver started at http://127.8.18.195:36511/ using document root <none> and password file <none>
I20260810 11:30:48.490188 13357 fs_manager.cc:359] Metadata directory not provided
I20260810 11:30:48.490296 13357 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:30:48.490679 13357 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:30:48.493992 13357 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786361401922362-8267-0/minicluster-data/ts-2/data/instance:
uuid: "1986cf0385cf406cbe9444e9747229db"
format_stamp: "Formatted at 2026-08-10 11:30:48 on dist-test-slave-srgv"
server_key: "e324cebed227f55fb257cb58c3cb1db9"
server_key_iv: "5537466bbef889aecc07036551444646"
server_key_version: "encryptionkey@0"
I20260810 11:30:48.494920 13357 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786361401922362-8267-0/minicluster-data/ts-2/wal/instance:
uuid: "1986cf0385cf406cbe9444e9747229db"
format_stamp: "Formatted at 2026-08-10 11:30:48 on dist-test-slave-srgv"
server_key: "e324cebed227f55fb257cb58c3cb1db9"
server_key_iv: "5537466bbef889aecc07036551444646"
server_key_version: "encryptionkey@0"
I20260810 11:30:48.501451 13357 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260810 11:30:48.505369 13371 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:30:48.507055 13357 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260810 11:30:48.507309 13357 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786361401922362-8267-0/minicluster-data/ts-2/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786361401922362-8267-0/minicluster-data/ts-2/wal
uuid: "1986cf0385cf406cbe9444e9747229db"
format_stamp: "Formatted at 2026-08-10 11:30:48 on dist-test-slave-srgv"
server_key: "e324cebed227f55fb257cb58c3cb1db9"
server_key_iv: "5537466bbef889aecc07036551444646"
server_key_version: "encryptionkey@0"
I20260810 11:30:48.507630 13357 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786361401922362-8267-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786361401922362-8267-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786361401922362-8267-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:30:48.514230 13357 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:30:48.515175 13357 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:30:48.515446 13357 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:30:48.516462 13357 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:30:48.518122 13357 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:30:48.518231 13357 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:30:48.518326 13357 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:30:48.518383 13357 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:30:48.555514 13357 rpc_server.cc:307] RPC server started. Bound to: 127.8.18.195:39393
I20260810 11:30:48.555560 13485 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.18.195:39393 every 8 connection(s)
I20260810 11:30:48.557067 13357 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786361401922362-8267-0/minicluster-data/ts-2/data/info.pb
I20260810 11:30:48.562716 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 13357
I20260810 11:30:48.562944 8267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786361401922362-8267-0/minicluster-data/ts-2/wal/instance
I20260810 11:30:48.563414 8267 external_mini_cluster.cc:1428] Setting key c90ee494f80ddf75987de172e9e13793
I20260810 11:30:48.566680 13219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:48.560091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.195:38747 (local address 127.8.18.254:34401)
0810 11:30:48.560314 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0810 11:30:48.560319 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:30:48.562100 (+ 1781us) server_negotiation.cc:407] Connection header received
0810 11:30:48.564133 (+ 2033us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:30:48.564139 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:30:48.564233 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:30:48.564374 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:30:48.565892 (+ 1518us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:30:48.565898 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:30:48.565903 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:30:48.565932 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:30:48.565967 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:30:48.565980 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:30:48.566111 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0810 11:30:48.566314 (+ 203us) server_negotiation.cc:299] Negotiation successful
0810 11:30:48.566390 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260810 11:30:48.567700 13488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:48.560706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:34401 (local address 127.8.18.195:38747)
0810 11:30:48.561892 (+ 1186us) negotiation.cc:107] Waiting for socket to connect
0810 11:30:48.561942 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0810 11:30:48.563796 (+ 1854us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:30:48.564632 (+ 836us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:30:48.564642 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:30:48.564759 (+ 117us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:30:48.565551 (+ 792us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 11:30:48.565554 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:30:48.565598 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 11:30:48.565695 (+ 97us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:30:48.566083 (+ 388us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:30:48.566094 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:30:48.566103 (+ 9us) client_negotiation.cc:769] Sending connection context
0810 11:30:48.566415 (+ 312us) client_negotiation.cc:240] Negotiation successful
0810 11:30:48.566714 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":598,"thread_start_us":376,"threads_started":1}
I20260810 11:30:48.569190 13486 heartbeater.cc:347] Connected to a master server at 127.8.18.254:34401
I20260810 11:30:48.569555 13486 heartbeater.cc:464] Registering TS with master...
I20260810 11:30:48.570168 13486 heartbeater.cc:511] Master 127.8.18.254:34401 requested a full tablet report, sending...
I20260810 11:30:48.571771 13024 ts_manager.cc:194] Registered new tserver with Master: 1986cf0385cf406cbe9444e9747229db (127.8.18.195:39393)
I20260810 11:30:48.579650 8267 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 11:30:48.587393 13219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:48.584535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44024 (local address 127.8.18.254:34401)
0810 11:30:48.584778 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0810 11:30:48.584788 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 11:30:48.585313 (+ 525us) server_negotiation.cc:407] Connection header received
0810 11:30:48.585527 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:30:48.585532 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:30:48.585589 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:30:48.585672 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:30:48.586686 (+ 1014us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:30:48.586690 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:30:48.586693 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:30:48.586710 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:30:48.586761 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:30:48.586773 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:30:48.586886 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0810 11:30:48.587022 (+ 136us) server_negotiation.cc:299] Negotiation successful
0810 11:30:48.587075 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260810 11:30:48.592684 13024 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44024:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 11:30:48.600404 13024 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 11:30:48.607607 13217 heartbeater.cc:503] Master 127.8.18.254:34401 was elected leader, sending a full tablet report...
I20260810 11:30:48.638774 13503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:48.616095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46948 (local address 127.8.18.194:43759)
0810 11:30:48.631653 (+ 15558us) server_negotiation.cc:206] Beginning negotiation
0810 11:30:48.631668 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 11:30:48.635137 (+ 3469us) server_negotiation.cc:407] Connection header received
0810 11:30:48.635472 (+ 335us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:30:48.635486 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:30:48.635819 (+ 333us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:30:48.636017 (+ 198us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:30:48.637613 (+ 1596us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:30:48.637625 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:30:48.637648 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:30:48.637683 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:30:48.637762 (+ 79us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:30:48.637787 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:30:48.637931 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0810 11:30:48.638251 (+ 320us) server_negotiation.cc:299] Negotiation successful
0810 11:30:48.638374 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15275,"thread_start_us":15574,"threads_started":1}
I20260810 11:30:48.639159 13504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:48.615319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.193:35711 (local address 127.0.0.1:34816)
0810 11:30:48.633042 (+ 17723us) negotiation.cc:107] Waiting for socket to connect
0810 11:30:48.633108 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0810 11:30:48.633546 (+ 438us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:30:48.636468 (+ 2922us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:30:48.636481 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:30:48.636511 (+ 30us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:30:48.637267 (+ 756us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 11:30:48.637271 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:30:48.637289 (+ 18us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 11:30:48.637349 (+ 60us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:30:48.638451 (+ 1102us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:30:48.638457 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:30:48.638462 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 11:30:48.638590 (+ 128us) client_negotiation.cc:240] Negotiation successful
0810 11:30:48.638689 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17329,"thread_start_us":19190,"threads_started":1}
I20260810 11:30:48.639644 13500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:48.615385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.194:43759 (local address 127.0.0.1:46948)
0810 11:30:48.634944 (+ 19559us) negotiation.cc:107] Waiting for socket to connect
0810 11:30:48.634977 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0810 11:30:48.635264 (+ 287us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:30:48.636071 (+ 807us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:30:48.636084 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:30:48.636191 (+ 107us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:30:48.637225 (+ 1034us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 11:30:48.637229 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:30:48.637261 (+ 32us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 11:30:48.637349 (+ 88us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:30:48.637997 (+ 648us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:30:48.638013 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:30:48.638026 (+ 13us) client_negotiation.cc:769] Sending connection context
0810 11:30:48.639082 (+ 1056us) client_negotiation.cc:240] Negotiation successful
0810 11:30:48.639154 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19338,"thread_start_us":19628,"threads_started":1}
I20260810 11:30:48.640071 13505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:48.620013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46392 (local address 127.8.18.195:39393)
0810 11:30:48.634270 (+ 14257us) server_negotiation.cc:206] Beginning negotiation
0810 11:30:48.634283 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 11:30:48.635580 (+ 1297us) server_negotiation.cc:407] Connection header received
0810 11:30:48.635885 (+ 305us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:30:48.635896 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:30:48.636150 (+ 254us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:30:48.636288 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:30:48.639014 (+ 2726us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:30:48.639025 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:30:48.639042 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:30:48.639062 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:30:48.639109 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:30:48.639130 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:30:48.639313 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0810 11:30:48.639569 (+ 256us) server_negotiation.cc:299] Negotiation successful
0810 11:30:48.639675 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14026,"thread_start_us":13830,"threads_started":1}
I20260810 11:30:48.640493 13502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:48.615389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34816 (local address 127.8.18.193:35711)
0810 11:30:48.635912 (+ 20523us) server_negotiation.cc:206] Beginning negotiation
0810 11:30:48.635922 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 11:30:48.635960 (+ 38us) server_negotiation.cc:407] Connection header received
0810 11:30:48.636032 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:30:48.636040 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:30:48.636262 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:30:48.636988 (+ 726us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:30:48.638176 (+ 1188us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:30:48.638186 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:30:48.638198 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:30:48.638218 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:30:48.638291 (+ 73us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:30:48.638312 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:30:48.639946 (+ 1634us) server_negotiation.cc:1035] Waiting for connection context
0810 11:30:48.640028 (+ 82us) server_negotiation.cc:299] Negotiation successful
0810 11:30:48.640135 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20341,"thread_start_us":20165,"threads_started":1}
I20260810 11:30:48.640921 13501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:48.619561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.195:39393 (local address 127.0.0.1:46392)
0810 11:30:48.635396 (+ 15835us) negotiation.cc:107] Waiting for socket to connect
0810 11:30:48.635422 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0810 11:30:48.635698 (+ 276us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:30:48.637620 (+ 1922us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:30:48.637627 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:30:48.637655 (+ 28us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:30:48.638436 (+ 781us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 11:30:48.638439 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:30:48.638468 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 11:30:48.638511 (+ 43us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:30:48.639378 (+ 867us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:30:48.639387 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:30:48.639395 (+ 8us) client_negotiation.cc:769] Sending connection context
0810 11:30:48.640428 (+ 1033us) client_negotiation.cc:240] Negotiation successful
0810 11:30:48.640475 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15653,"thread_start_us":12860,"threads_started":1}
I20260810 11:30:48.644229 13285 tablet_service.cc:1467] Processing CreateTablet for tablet 939c9a077b8b43128f59a2e848e61d7a (DEFAULT_TABLE table=test-table [id=334eba46792a4cffa2944d5c95f4ac4c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:30:48.644479 13148 tablet_service.cc:1467] Processing CreateTablet for tablet 939c9a077b8b43128f59a2e848e61d7a (DEFAULT_TABLE table=test-table [id=334eba46792a4cffa2944d5c95f4ac4c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:30:48.644479 13419 tablet_service.cc:1467] Processing CreateTablet for tablet 939c9a077b8b43128f59a2e848e61d7a (DEFAULT_TABLE table=test-table [id=334eba46792a4cffa2944d5c95f4ac4c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:30:48.645880 13285 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 939c9a077b8b43128f59a2e848e61d7a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:30:48.645957 13419 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 939c9a077b8b43128f59a2e848e61d7a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:30:48.645972 13148 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 939c9a077b8b43128f59a2e848e61d7a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:30:48.659158 13506 tablet_bootstrap.cc:492] T 939c9a077b8b43128f59a2e848e61d7a P de9c26202bc14312b1c84312997acc41: Bootstrap starting.
I20260810 11:30:48.663594 13506 tablet_bootstrap.cc:654] T 939c9a077b8b43128f59a2e848e61d7a P de9c26202bc14312b1c84312997acc41: Neither blocks nor log segments found. Creating new log.
I20260810 11:30:48.664886 13506 log.cc:824] T 939c9a077b8b43128f59a2e848e61d7a P de9c26202bc14312b1c84312997acc41: Log is configured to *not* fsync() on all Append() calls
I20260810 11:30:48.668085 13508 tablet_bootstrap.cc:492] T 939c9a077b8b43128f59a2e848e61d7a P 1986cf0385cf406cbe9444e9747229db: Bootstrap starting.
I20260810 11:30:48.668953 13506 tablet_bootstrap.cc:492] T 939c9a077b8b43128f59a2e848e61d7a P de9c26202bc14312b1c84312997acc41: No bootstrap required, opened a new log
I20260810 11:30:48.669262 13506 ts_tablet_manager.cc:1397] T 939c9a077b8b43128f59a2e848e61d7a P de9c26202bc14312b1c84312997acc41: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.005s
I20260810 11:30:48.671301 13507 tablet_bootstrap.cc:492] T 939c9a077b8b43128f59a2e848e61d7a P 31228a8ad6f6423f80644ea244c6fa24: Bootstrap starting.
I20260810 11:30:48.671651 13508 tablet_bootstrap.cc:654] T 939c9a077b8b43128f59a2e848e61d7a P 1986cf0385cf406cbe9444e9747229db: Neither blocks nor log segments found. Creating new log.
I20260810 11:30:48.673269 13508 log.cc:824] T 939c9a077b8b43128f59a2e848e61d7a P 1986cf0385cf406cbe9444e9747229db: Log is configured to *not* fsync() on all Append() calls
I20260810 11:30:48.675762 13507 tablet_bootstrap.cc:654] T 939c9a077b8b43128f59a2e848e61d7a P 31228a8ad6f6423f80644ea244c6fa24: Neither blocks nor log segments found. Creating new log.
I20260810 11:30:48.677515 13507 log.cc:824] T 939c9a077b8b43128f59a2e848e61d7a P 31228a8ad6f6423f80644ea244c6fa24: Log is configured to *not* fsync() on all Append() calls
I20260810 11:30:48.680166 13506 raft_consensus.cc:348] T 939c9a077b8b43128f59a2e848e61d7a P de9c26202bc14312b1c84312997acc41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31228a8ad6f6423f80644ea244c6fa24" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 35711 } } peers { permanent_uuid: "de9c26202bc14312b1c84312997acc41" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 43759 } } peers { permanent_uuid: "1986cf0385cf406cbe9444e9747229db" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 39393 } }
I20260810 11:30:48.680728 13506 raft_consensus.cc:374] T 939c9a077b8b43128f59a2e848e61d7a P de9c26202bc14312b1c84312997acc41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:30:48.680840 13506 raft_consensus.cc:729] T 939c9a077b8b43128f59a2e848e61d7a P de9c26202bc14312b1c84312997acc41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de9c26202bc14312b1c84312997acc41, State: Initialized, Role: FOLLOWER
I20260810 11:30:48.681504 13507 tablet_bootstrap.cc:492] T 939c9a077b8b43128f59a2e848e61d7a P 31228a8ad6f6423f80644ea244c6fa24: No bootstrap required, opened a new log
I20260810 11:30:48.681519 13508 tablet_bootstrap.cc:492] T 939c9a077b8b43128f59a2e848e61d7a P 1986cf0385cf406cbe9444e9747229db: No bootstrap required, opened a new log
I20260810 11:30:48.681849 13508 ts_tablet_manager.cc:1397] T 939c9a077b8b43128f59a2e848e61d7a P 1986cf0385cf406cbe9444e9747229db: Time spent bootstrapping tablet: real 0.014s user 0.005s sys 0.003s
I20260810 11:30:48.681671 13506 consensus_queue.cc:260] T 939c9a077b8b43128f59a2e848e61d7a P de9c26202bc14312b1c84312997acc41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31228a8ad6f6423f80644ea244c6fa24" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 35711 } } peers { permanent_uuid: "de9c26202bc14312b1c84312997acc41" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 43759 } } peers { permanent_uuid: "1986cf0385cf406cbe9444e9747229db" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 39393 } }
I20260810 11:30:48.681941 13507 ts_tablet_manager.cc:1397] T 939c9a077b8b43128f59a2e848e61d7a P 31228a8ad6f6423f80644ea244c6fa24: Time spent bootstrapping tablet: real 0.016s user 0.010s sys 0.000s
I20260810 11:30:48.684592 13354 heartbeater.cc:503] Master 127.8.18.254:34401 was elected leader, sending a full tablet report...
I20260810 11:30:48.685391 13506 ts_tablet_manager.cc:1428] T 939c9a077b8b43128f59a2e848e61d7a P de9c26202bc14312b1c84312997acc41: Time spent starting tablet: real 0.016s user 0.015s sys 0.000s
I20260810 11:30:48.691792 13507 raft_consensus.cc:348] T 939c9a077b8b43128f59a2e848e61d7a P 31228a8ad6f6423f80644ea244c6fa24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31228a8ad6f6423f80644ea244c6fa24" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 35711 } } peers { permanent_uuid: "de9c26202bc14312b1c84312997acc41" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 43759 } } peers { permanent_uuid: "1986cf0385cf406cbe9444e9747229db" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 39393 } }
I20260810 11:30:48.692273 13507 raft_consensus.cc:374] T 939c9a077b8b43128f59a2e848e61d7a P 31228a8ad6f6423f80644ea244c6fa24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:30:48.692397 13507 raft_consensus.cc:729] T 939c9a077b8b43128f59a2e848e61d7a P 31228a8ad6f6423f80644ea244c6fa24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31228a8ad6f6423f80644ea244c6fa24, State: Initialized, Role: FOLLOWER
I20260810 11:30:48.692783 13508 raft_consensus.cc:348] T 939c9a077b8b43128f59a2e848e61d7a P 1986cf0385cf406cbe9444e9747229db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31228a8ad6f6423f80644ea244c6fa24" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 35711 } } peers { permanent_uuid: "de9c26202bc14312b1c84312997acc41" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 43759 } } peers { permanent_uuid: "1986cf0385cf406cbe9444e9747229db" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 39393 } }
I20260810 11:30:48.693192 13508 raft_consensus.cc:374] T 939c9a077b8b43128f59a2e848e61d7a P 1986cf0385cf406cbe9444e9747229db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:30:48.693035 13507 consensus_queue.cc:260] T 939c9a077b8b43128f59a2e848e61d7a P 31228a8ad6f6423f80644ea244c6fa24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31228a8ad6f6423f80644ea244c6fa24" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 35711 } } peers { permanent_uuid: "de9c26202bc14312b1c84312997acc41" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 43759 } } peers { permanent_uuid: "1986cf0385cf406cbe9444e9747229db" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 39393 } }
I20260810 11:30:48.693302 13508 raft_consensus.cc:729] T 939c9a077b8b43128f59a2e848e61d7a P 1986cf0385cf406cbe9444e9747229db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1986cf0385cf406cbe9444e9747229db, State: Initialized, Role: FOLLOWER
I20260810 11:30:48.693922 13508 consensus_queue.cc:260] T 939c9a077b8b43128f59a2e848e61d7a P 1986cf0385cf406cbe9444e9747229db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31228a8ad6f6423f80644ea244c6fa24" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 35711 } } peers { permanent_uuid: "de9c26202bc14312b1c84312997acc41" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 43759 } } peers { permanent_uuid: "1986cf0385cf406cbe9444e9747229db" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 39393 } }
I20260810 11:30:48.695075 13507 ts_tablet_manager.cc:1428] T 939c9a077b8b43128f59a2e848e61d7a P 31228a8ad6f6423f80644ea244c6fa24: Time spent starting tablet: real 0.013s user 0.012s sys 0.000s
I20260810 11:30:48.695072 13486 heartbeater.cc:503] Master 127.8.18.254:34401 was elected leader, sending a full tablet report...
I20260810 11:30:48.695731 13508 ts_tablet_manager.cc:1428] T 939c9a077b8b43128f59a2e848e61d7a P 1986cf0385cf406cbe9444e9747229db: Time spent starting tablet: real 0.014s user 0.009s sys 0.004s
W20260810 11:30:48.809134 13487 tablet.cc:2369] T 939c9a077b8b43128f59a2e848e61d7a P 1986cf0385cf406cbe9444e9747229db: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 11:30:48.811233 13513 raft_consensus.cc:482] T 939c9a077b8b43128f59a2e848e61d7a P 31228a8ad6f6423f80644ea244c6fa24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 11:30:48.811622 13513 raft_consensus.cc:504] T 939c9a077b8b43128f59a2e848e61d7a P 31228a8ad6f6423f80644ea244c6fa24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31228a8ad6f6423f80644ea244c6fa24" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 35711 } } peers { permanent_uuid: "de9c26202bc14312b1c84312997acc41" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 43759 } } peers { permanent_uuid: "1986cf0385cf406cbe9444e9747229db" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 39393 } }
I20260810 11:30:48.813562 13513 leader_election.cc:288] T 939c9a077b8b43128f59a2e848e61d7a P 31228a8ad6f6423f80644ea244c6fa24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers de9c26202bc14312b1c84312997acc41 (127.8.18.194:43759), 1986cf0385cf406cbe9444e9747229db (127.8.18.195:39393)
I20260810 11:30:48.817239 13505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:48.814087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.193:53667 (local address 127.8.18.195:39393)
0810 11:30:48.814333 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0810 11:30:48.814337 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 11:30:48.815052 (+ 715us) server_negotiation.cc:407] Connection header received
0810 11:30:48.815260 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:30:48.815269 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:30:48.815344 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:30:48.815455 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:30:48.816564 (+ 1109us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:30:48.816570 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:30:48.816574 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:30:48.816606 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:30:48.816639 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:30:48.816655 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:30:48.816804 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0810 11:30:48.816880 (+ 76us) server_negotiation.cc:299] Negotiation successful
0810 11:30:48.816957 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135}
I20260810 11:30:48.817462 13503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:48.813967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.193:40619 (local address 127.8.18.194:43759)
0810 11:30:48.814193 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0810 11:30:48.814201 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:30:48.815201 (+ 1000us) server_negotiation.cc:407] Connection header received
0810 11:30:48.815386 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:30:48.815390 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:30:48.815440 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:30:48.815553 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:30:48.816777 (+ 1224us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:30:48.816781 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:30:48.816784 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:30:48.816801 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:30:48.816831 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:30:48.816843 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:30:48.816980 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0810 11:30:48.817062 (+ 82us) server_negotiation.cc:299] Negotiation successful
0810 11:30:48.817141 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260810 11:30:48.817667 13516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:48.813954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.194:43759 (local address 127.8.18.193:40619)
0810 11:30:48.815068 (+ 1114us) negotiation.cc:107] Waiting for socket to connect
0810 11:30:48.815083 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 11:30:48.815229 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:30:48.815575 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:30:48.815593 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:30:48.815618 (+ 25us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:30:48.816532 (+ 914us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 11:30:48.816534 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:30:48.816548 (+ 14us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 11:30:48.816590 (+ 42us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:30:48.816949 (+ 359us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:30:48.816954 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:30:48.816959 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 11:30:48.817123 (+ 164us) client_negotiation.cc:240] Negotiation successful
0810 11:30:48.817172 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":994,"mutex_wait_us":34,"thread_start_us":701,"threads_started":1}
I20260810 11:30:48.818033 13515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:48.813949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.195:39393 (local address 127.8.18.193:53667)
0810 11:30:48.814914 (+ 965us) negotiation.cc:107] Waiting for socket to connect
0810 11:30:48.814932 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 11:30:48.815117 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:30:48.815484 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:30:48.815488 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:30:48.815514 (+ 26us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:30:48.816342 (+ 828us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 11:30:48.816345 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:30:48.816359 (+ 14us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 11:30:48.816401 (+ 42us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:30:48.816770 (+ 369us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:30:48.816777 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:30:48.816784 (+ 7us) client_negotiation.cc:769] Sending connection context
0810 11:30:48.816930 (+ 146us) client_negotiation.cc:240] Negotiation successful
0810 11:30:48.816983 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":807,"thread_start_us":593,"threads_started":1}
I20260810 11:30:48.818982 13440 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "939c9a077b8b43128f59a2e848e61d7a" candidate_uuid: "31228a8ad6f6423f80644ea244c6fa24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1986cf0385cf406cbe9444e9747229db" is_pre_election: true
I20260810 11:30:48.818908 13306 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "939c9a077b8b43128f59a2e848e61d7a" candidate_uuid: "31228a8ad6f6423f80644ea244c6fa24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de9c26202bc14312b1c84312997acc41" is_pre_election: true
I20260810 11:30:48.819500 13440 raft_consensus.cc:2454] T 939c9a077b8b43128f59a2e848e61d7a P 1986cf0385cf406cbe9444e9747229db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31228a8ad6f6423f80644ea244c6fa24 in term 0.
I20260810 11:30:48.819499 13306 raft_consensus.cc:2454] T 939c9a077b8b43128f59a2e848e61d7a P de9c26202bc14312b1c84312997acc41 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31228a8ad6f6423f80644ea244c6fa24 in term 0.
I20260810 11:30:48.820223 13103 leader_election.cc:302] T 939c9a077b8b43128f59a2e848e61d7a P 31228a8ad6f6423f80644ea244c6fa24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31228a8ad6f6423f80644ea244c6fa24, de9c26202bc14312b1c84312997acc41; no voters:
I20260810 11:30:48.820672 13513 raft_consensus.cc:2781] T 939c9a077b8b43128f59a2e848e61d7a P 31228a8ad6f6423f80644ea244c6fa24 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 11:30:48.820858 13513 raft_consensus.cc:482] T 939c9a077b8b43128f59a2e848e61d7a P 31228a8ad6f6423f80644ea244c6fa24 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 11:30:48.820971 13513 raft_consensus.cc:3037] T 939c9a077b8b43128f59a2e848e61d7a P 31228a8ad6f6423f80644ea244c6fa24 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:30:48.823442 13513 raft_consensus.cc:504] T 939c9a077b8b43128f59a2e848e61d7a P 31228a8ad6f6423f80644ea244c6fa24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31228a8ad6f6423f80644ea244c6fa24" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 35711 } } peers { permanent_uuid: "de9c26202bc14312b1c84312997acc41" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 43759 } } peers { permanent_uuid: "1986cf0385cf406cbe9444e9747229db" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 39393 } }
I20260810 11:30:48.824223 13513 leader_election.cc:288] T 939c9a077b8b43128f59a2e848e61d7a P 31228a8ad6f6423f80644ea244c6fa24 [CANDIDATE]: Term 1 election: Requested vote from peers de9c26202bc14312b1c84312997acc41 (127.8.18.194:43759), 1986cf0385cf406cbe9444e9747229db (127.8.18.195:39393)
I20260810 11:30:48.824797 13306 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "939c9a077b8b43128f59a2e848e61d7a" candidate_uuid: "31228a8ad6f6423f80644ea244c6fa24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de9c26202bc14312b1c84312997acc41"
I20260810 11:30:48.824918 13440 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "939c9a077b8b43128f59a2e848e61d7a" candidate_uuid: "31228a8ad6f6423f80644ea244c6fa24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1986cf0385cf406cbe9444e9747229db"
I20260810 11:30:48.825063 13306 raft_consensus.cc:3037] T 939c9a077b8b43128f59a2e848e61d7a P de9c26202bc14312b1c84312997acc41 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:30:48.825179 13440 raft_consensus.cc:3037] T 939c9a077b8b43128f59a2e848e61d7a P 1986cf0385cf406cbe9444e9747229db [term 0 FOLLOWER]: Advancing to term 1
W20260810 11:30:48.826504 13355 tablet.cc:2369] T 939c9a077b8b43128f59a2e848e61d7a P de9c26202bc14312b1c84312997acc41: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 11:30:48.828166 13440 raft_consensus.cc:2454] T 939c9a077b8b43128f59a2e848e61d7a P 1986cf0385cf406cbe9444e9747229db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31228a8ad6f6423f80644ea244c6fa24 in term 1.
I20260810 11:30:48.828733 13306 raft_consensus.cc:2454] T 939c9a077b8b43128f59a2e848e61d7a P de9c26202bc14312b1c84312997acc41 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31228a8ad6f6423f80644ea244c6fa24 in term 1.
I20260810 11:30:48.828809 13104 leader_election.cc:302] T 939c9a077b8b43128f59a2e848e61d7a P 31228a8ad6f6423f80644ea244c6fa24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1986cf0385cf406cbe9444e9747229db, 31228a8ad6f6423f80644ea244c6fa24; no voters:
I20260810 11:30:48.829253 13513 raft_consensus.cc:2781] T 939c9a077b8b43128f59a2e848e61d7a P 31228a8ad6f6423f80644ea244c6fa24 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:30:48.829803 13513 raft_consensus.cc:686] T 939c9a077b8b43128f59a2e848e61d7a P 31228a8ad6f6423f80644ea244c6fa24 [term 1 LEADER]: Becoming Leader. State: Replica: 31228a8ad6f6423f80644ea244c6fa24, State: Running, Role: LEADER
I20260810 11:30:48.830327 13513 consensus_queue.cc:237] T 939c9a077b8b43128f59a2e848e61d7a P 31228a8ad6f6423f80644ea244c6fa24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31228a8ad6f6423f80644ea244c6fa24" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 35711 } } peers { permanent_uuid: "de9c26202bc14312b1c84312997acc41" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 43759 } } peers { permanent_uuid: "1986cf0385cf406cbe9444e9747229db" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 39393 } }
I20260810 11:30:48.835820 13023 catalog_manager.cc:5495] T 939c9a077b8b43128f59a2e848e61d7a P 31228a8ad6f6423f80644ea244c6fa24 reported cstate change: term changed from 0 to 1, leader changed from <none> to 31228a8ad6f6423f80644ea244c6fa24 (127.8.18.193). New cstate: current_term: 1 leader_uuid: "31228a8ad6f6423f80644ea244c6fa24" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31228a8ad6f6423f80644ea244c6fa24" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 35711 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de9c26202bc14312b1c84312997acc41" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 43759 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1986cf0385cf406cbe9444e9747229db" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 39393 } health_report { overall_health: UNKNOWN } } }
W20260810 11:30:48.841205 13218 tablet.cc:2369] T 939c9a077b8b43128f59a2e848e61d7a P 31228a8ad6f6423f80644ea244c6fa24: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 11:30:48.885051 13502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:48.882532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34820 (local address 127.8.18.193:35711)
0810 11:30:48.882807 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0810 11:30:48.882812 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:30:48.882859 (+ 47us) server_negotiation.cc:407] Connection header received
0810 11:30:48.883083 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:30:48.883087 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:30:48.883176 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:30:48.883282 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:30:48.884361 (+ 1079us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:30:48.884365 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:30:48.884368 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:30:48.884385 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:30:48.884420 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:30:48.884432 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:30:48.884541 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0810 11:30:48.884670 (+ 129us) server_negotiation.cc:299] Negotiation successful
0810 11:30:48.884720 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106}
I20260810 11:30:48.895046 13440 raft_consensus.cc:1261] T 939c9a077b8b43128f59a2e848e61d7a P 1986cf0385cf406cbe9444e9747229db [term 1 FOLLOWER]: Refusing update from remote peer 31228a8ad6f6423f80644ea244c6fa24: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 11:30:48.895077 13306 raft_consensus.cc:1261] T 939c9a077b8b43128f59a2e848e61d7a P de9c26202bc14312b1c84312997acc41 [term 1 FOLLOWER]: Refusing update from remote peer 31228a8ad6f6423f80644ea244c6fa24: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 11:30:48.896173 13513 consensus_queue.cc:1035] T 939c9a077b8b43128f59a2e848e61d7a P 31228a8ad6f6423f80644ea244c6fa24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1986cf0385cf406cbe9444e9747229db" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 39393 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 11:30:48.896590 13517 consensus_queue.cc:1035] T 939c9a077b8b43128f59a2e848e61d7a P 31228a8ad6f6423f80644ea244c6fa24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de9c26202bc14312b1c84312997acc41" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 43759 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 11:30:48.911819 13520 mvcc.cc:205] Tried to move back new op lower bound from 7316936494661636096 to 7316936494413398016. Current Snapshot: MvccSnapshot[applied={T|T < 7316936494661636096}]
I20260810 11:30:48.928815 13023 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44024:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 11:30:48.929278 13023 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44024:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 11:30:48.933748 13023 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 324304b5aacc45239b49bb96dae05f6c: Sending DeleteTablet for 3 replicas of tablet 939c9a077b8b43128f59a2e848e61d7a
I20260810 11:30:48.935250 13285 tablet_service.cc:1514] Processing DeleteTablet for tablet 939c9a077b8b43128f59a2e848e61d7a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:30:48 UTC) from {username='slave'} at 127.0.0.1:46948
I20260810 11:30:48.935247 13148 tablet_service.cc:1514] Processing DeleteTablet for tablet 939c9a077b8b43128f59a2e848e61d7a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:30:48 UTC) from {username='slave'} at 127.0.0.1:34816
I20260810 11:30:48.935554 13419 tablet_service.cc:1514] Processing DeleteTablet for tablet 939c9a077b8b43128f59a2e848e61d7a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:30:48 UTC) from {username='slave'} at 127.0.0.1:46392
I20260810 11:30:48.938726 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 13085
I20260810 11:30:48.949919 13529 tablet_replica.cc:330] T 939c9a077b8b43128f59a2e848e61d7a P de9c26202bc14312b1c84312997acc41: stopping tablet replica
I20260810 11:30:48.950659 13530 tablet_replica.cc:330] T 939c9a077b8b43128f59a2e848e61d7a P 31228a8ad6f6423f80644ea244c6fa24: stopping tablet replica
I20260810 11:30:48.951850 13530 raft_consensus.cc:2229] T 939c9a077b8b43128f59a2e848e61d7a P 31228a8ad6f6423f80644ea244c6fa24 [term 1 LEADER]: Raft consensus shutting down.
I20260810 11:30:48.951822 13529 raft_consensus.cc:2229] T 939c9a077b8b43128f59a2e848e61d7a P de9c26202bc14312b1c84312997acc41 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 11:30:48.952554 13529 raft_consensus.cc:2258] T 939c9a077b8b43128f59a2e848e61d7a P de9c26202bc14312b1c84312997acc41 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:30:48.952906 13530 raft_consensus.cc:2258] T 939c9a077b8b43128f59a2e848e61d7a P 31228a8ad6f6423f80644ea244c6fa24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:30:48.955024 13502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:48.940325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34836 (local address 127.8.18.193:35711)
0810 11:30:48.940604 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0810 11:30:48.940614 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 11:30:48.946328 (+ 5714us) server_negotiation.cc:407] Connection header received
0810 11:30:48.946654 (+ 326us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:30:48.946664 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:30:48.946769 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:30:48.946929 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:30:48.953720 (+ 6791us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:30:48.953731 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:30:48.953738 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:30:48.953788 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:30:48.953858 (+ 70us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:30:48.953892 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:30:48.954068 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0810 11:30:48.954453 (+ 385us) server_negotiation.cc:299] Negotiation successful
0810 11:30:48.954544 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260810 11:30:48.955806 13530 ts_tablet_manager.cc:1905] T 939c9a077b8b43128f59a2e848e61d7a P 31228a8ad6f6423f80644ea244c6fa24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:30:48.956539 13212 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:30:48.956970 13531 tablet_replica.cc:330] T 939c9a077b8b43128f59a2e848e61d7a P 1986cf0385cf406cbe9444e9747229db: stopping tablet replica
I20260810 11:30:48.957507 13531 raft_consensus.cc:2229] T 939c9a077b8b43128f59a2e848e61d7a P 1986cf0385cf406cbe9444e9747229db [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 11:30:48.958097 13531 raft_consensus.cc:2258] T 939c9a077b8b43128f59a2e848e61d7a P 1986cf0385cf406cbe9444e9747229db [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:30:48.959211 13529 ts_tablet_manager.cc:1905] T 939c9a077b8b43128f59a2e848e61d7a P de9c26202bc14312b1c84312997acc41: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:30:48.961285 13531 ts_tablet_manager.cc:1905] T 939c9a077b8b43128f59a2e848e61d7a P 1986cf0385cf406cbe9444e9747229db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:30:48.966959 13529 ts_tablet_manager.cc:1918] T 939c9a077b8b43128f59a2e848e61d7a P de9c26202bc14312b1c84312997acc41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 11:30:48.967193 13529 log.cc:1196] T 939c9a077b8b43128f59a2e848e61d7a P de9c26202bc14312b1c84312997acc41: Deleting WAL directory at /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786361401922362-8267-0/minicluster-data/ts-1/wal/wals/939c9a077b8b43128f59a2e848e61d7a
I20260810 11:30:48.967782 13529 ts_tablet_manager.cc:1939] T 939c9a077b8b43128f59a2e848e61d7a P de9c26202bc14312b1c84312997acc41: Deleting consensus metadata
I20260810 11:30:48.969326 13531 ts_tablet_manager.cc:1918] T 939c9a077b8b43128f59a2e848e61d7a P 1986cf0385cf406cbe9444e9747229db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 11:30:48.969588 13531 log.cc:1196] T 939c9a077b8b43128f59a2e848e61d7a P 1986cf0385cf406cbe9444e9747229db: Deleting WAL directory at /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786361401922362-8267-0/minicluster-data/ts-2/wal/wals/939c9a077b8b43128f59a2e848e61d7a
I20260810 11:30:48.969996 13010 catalog_manager.cc:4841] TS de9c26202bc14312b1c84312997acc41 (127.8.18.194:43759): tablet 939c9a077b8b43128f59a2e848e61d7a (table test-table [id=334eba46792a4cffa2944d5c95f4ac4c]) successfully deleted
I20260810 11:30:48.970242 13531 ts_tablet_manager.cc:1939] T 939c9a077b8b43128f59a2e848e61d7a P 1986cf0385cf406cbe9444e9747229db: Deleting consensus metadata
I20260810 11:30:48.972429 13011 catalog_manager.cc:4841] TS 1986cf0385cf406cbe9444e9747229db (127.8.18.195:39393): tablet 939c9a077b8b43128f59a2e848e61d7a (table test-table [id=334eba46792a4cffa2944d5c95f4ac4c]) successfully deleted
I20260810 11:30:49.055655 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 13085
W20260810 11:30:49.077051 13009 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.18.193:35711 (error 108)
I20260810 11:30:49.077363 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 13221
I20260810 11:30:49.079638 13501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:49.078999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.193:35711 (local address 127.0.0.1:34840)
0810 11:30:49.079235 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0810 11:30:49.079461 (+ 226us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.193:35711: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
W20260810 11:30:49.079954 13009 catalog_manager.cc:4584] TS 31228a8ad6f6423f80644ea244c6fa24 (127.8.18.193:35711): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 939c9a077b8b43128f59a2e848e61d7a: Network error: Client connection negotiation failed: client connection to 127.8.18.193:35711: connect: Connection refused (error 111)
I20260810 11:30:49.081002 13503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:49.078640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46960 (local address 127.8.18.194:43759)
0810 11:30:49.078852 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0810 11:30:49.078860 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:30:49.078881 (+ 21us) server_negotiation.cc:407] Connection header received
0810 11:30:49.078944 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:30:49.078949 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:30:49.079083 (+ 134us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:30:49.079193 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:30:49.080268 (+ 1075us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:30:49.080276 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:30:49.080281 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:30:49.080300 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:30:49.080342 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:30:49.080359 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:30:49.080511 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0810 11:30:49.080647 (+ 136us) server_negotiation.cc:299] Negotiation successful
0810 11:30:49.080715 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260810 11:30:49.081845 13348 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:30:49.133888 13501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:49.133357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.193:35711 (local address 127.0.0.1:34856)
0810 11:30:49.133618 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0810 11:30:49.133715 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.193:35711: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117}
I20260810 11:30:49.135032 13501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:49.134652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.193:35711 (local address 127.0.0.1:34872)
0810 11:30:49.134829 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0810 11:30:49.134903 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.193:35711: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260810 11:30:49.174470 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 13221
I20260810 11:30:49.192173 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 13357
I20260810 11:30:49.195612 13505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:49.193136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46400 (local address 127.8.18.195:39393)
0810 11:30:49.193333 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0810 11:30:49.193339 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:30:49.193361 (+ 22us) server_negotiation.cc:407] Connection header received
0810 11:30:49.193617 (+ 256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:30:49.193659 (+ 42us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:30:49.193748 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:30:49.193842 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:30:49.194875 (+ 1033us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:30:49.194881 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:30:49.194885 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:30:49.194906 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:30:49.194949 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:30:49.194965 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:30:49.195127 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0810 11:30:49.195239 (+ 112us) server_negotiation.cc:299] Negotiation successful
0810 11:30:49.195306 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260810 11:30:49.195891 13501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:49.195330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.193:35711 (local address 127.0.0.1:34888)
0810 11:30:49.195590 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0810 11:30:49.195676 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.193:35711: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144}
I20260810 11:30:49.196416 13481 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:30:49.197208 13501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:49.196722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.193:35711 (local address 127.0.0.1:34898)
0810 11:30:49.196929 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0810 11:30:49.197031 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.193:35711: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260810 11:30:49.278656 13501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:49.277989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.193:35711 (local address 127.0.0.1:34912)
0810 11:30:49.278274 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0810 11:30:49.278415 (+ 141us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.193:35711: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116}
I20260810 11:30:49.279973 13501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:49.279473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.193:35711 (local address 127.0.0.1:34916)
0810 11:30:49.279715 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0810 11:30:49.279808 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.193:35711: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124}
I20260810 11:30:49.289942 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 13357
I20260810 11:30:49.307708 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 12991
I20260810 11:30:49.309119 13055 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:30:49.424544 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 12991
2026-08-10T11:30:49Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (2997 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260810 11:30:49.470861 8267 mini_oidc.cc:152] Starting JWKS server
I20260810 11:30:49.474179 8267 webserver.cc:460] Webserver started at https://127.0.0.1:33725/ using document root <none> and password file <none>
I20260810 11:30:49.474689 8267 mini_oidc.cc:189] Starting OIDC Discovery server
I20260810 11:30:49.475538 8267 webserver.cc:460] Webserver started at http://127.0.0.1:43945/ using document root <none> and password file <none>
2026-08-10T11:30:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T11:30:49Z Disabled control of system clock
I20260810 11:30:49.511133 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786361401922362-8267-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786361401922362-8267-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786361401922362-8267-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.18.254:43199
--webserver_interface=127.8.18.254
--webserver_port=0
--builtin_ntp_servers=127.8.18.212:43553
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.18.254:43199
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33725/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786361401922362-8267-0/testchainca.cert with env {}
W20260810 11:30:49.875311 13543 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:30:49.875771 13543 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:30:49.875965 13543 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:30:49.884600 13543 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 11:30:49.884794 13543 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:30:49.884887 13543 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:30:49.884985 13543 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 11:30:49.885106 13543 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260810 11:30:49.885242 13543 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260810 11:30:49.885314 13543 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:33725/jwks.json
I20260810 11:30:49.896786 13543 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.18.212:43553
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786361401922362-8267-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786361401922362-8267-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.18.254:43199
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.18.254:43199
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33725/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.18.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786361401922362-8267-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786361401922362-8267-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:30:49.898399 13543 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:30:49.899938 13543 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:30:49.913455 13548 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:30:49.913528 13551 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:30:49.913496 13549 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:30:49.914506 13543 server_base.cc:1034] running on GCE node
I20260810 11:30:49.915428 13543 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:30:49.917101 13543 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:30:49.918380 13543 hybrid_clock.cc:648] HybridClock initialized: now 1786361449918312 us; error 99 us; skew 500 ppm
I20260810 11:30:49.918818 13543 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 11:30:49.921702 13543 webserver.cc:460] Webserver started at http://127.8.18.254:39239/ using document root <none> and password file <none>
I20260810 11:30:49.922540 13543 fs_manager.cc:359] Metadata directory not provided
I20260810 11:30:49.922701 13543 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:30:49.923045 13543 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:30:49.926421 13543 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786361401922362-8267-0/minicluster-data/master-0/data/instance:
uuid: "f300c114e8b047f2a36a606ddf2c51a4"
format_stamp: "Formatted at 2026-08-10 11:30:49 on dist-test-slave-srgv"
server_key: "75aab914e9b5e34033827ac428839e74"
server_key_iv: "677e275dd38241c2ad689c409ce23562"
server_key_version: "encryptionkey@0"
I20260810 11:30:49.927364 13543 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786361401922362-8267-0/minicluster-data/master-0/wal/instance:
uuid: "f300c114e8b047f2a36a606ddf2c51a4"
format_stamp: "Formatted at 2026-08-10 11:30:49 on dist-test-slave-srgv"
server_key: "75aab914e9b5e34033827ac428839e74"
server_key_iv: "677e275dd38241c2ad689c409ce23562"
server_key_version: "encryptionkey@0"
I20260810 11:30:49.933650 13543 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.005s
I20260810 11:30:49.938305 13557 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:30:49.940248 13543 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260810 11:30:49.940548 13543 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786361401922362-8267-0/minicluster-data/master-0/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786361401922362-8267-0/minicluster-data/master-0/wal
uuid: "f300c114e8b047f2a36a606ddf2c51a4"
format_stamp: "Formatted at 2026-08-10 11:30:49 on dist-test-slave-srgv"
server_key: "75aab914e9b5e34033827ac428839e74"
server_key_iv: "677e275dd38241c2ad689c409ce23562"
server_key_version: "encryptionkey@0"
I20260810 11:30:49.940860 13543 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786361401922362-8267-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786361401922362-8267-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786361401922362-8267-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:30:49.959096 13543 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:30:49.960299 13543 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:30:49.960757 13543 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:30:49.983160 13610 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.18.254:43199 every 8 connection(s)
I20260810 11:30:49.983235 13543 rpc_server.cc:307] RPC server started. Bound to: 127.8.18.254:43199
I20260810 11:30:49.984931 13543 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
I20260810 11:30:49.987599 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 13543
I20260810 11:30:49.987823 8267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786361401922362-8267-0/minicluster-data/master-0/wal/instance
I20260810 11:30:49.988698 8267 external_mini_cluster.cc:1428] Setting key 5f80933ec39fc96a19a850ee02a9b45e
I20260810 11:30:49.990568 13611 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:30:50.005241 13614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:49.991195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42006 (local address 127.8.18.254:43199)
0810 11:30:49.992346 (+ 1151us) server_negotiation.cc:206] Beginning negotiation
0810 11:30:49.992368 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0810 11:30:49.992418 (+ 50us) server_negotiation.cc:407] Connection header received
0810 11:30:49.994276 (+ 1858us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:30:49.994305 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:30:49.994926 (+ 621us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:30:49.995343 (+ 417us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:30:49.996264 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:49.997828 (+ 1564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:49.999914 (+ 2086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:50.000673 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:50.000827 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:50.002441 (+ 1614us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:30:50.002450 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:30:50.002468 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:30:50.003166 (+ 698us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:30:50.003254 (+ 88us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:30:50.003282 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:30:50.003415 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0810 11:30:50.003688 (+ 273us) server_negotiation.cc:299] Negotiation successful
0810 11:30:50.004122 (+ 434us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":679,"thread_start_us":455,"threads_started":1}
I20260810 11:30:50.006403 13611 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f300c114e8b047f2a36a606ddf2c51a4: Bootstrap starting.
I20260810 11:30:50.009913 8267 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 11:30:50.010311 13611 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f300c114e8b047f2a36a606ddf2c51a4: Neither blocks nor log segments found. Creating new log.
I20260810 11:30:50.011428 13611 log.cc:824] T 00000000000000000000000000000000 P f300c114e8b047f2a36a606ddf2c51a4: Log is configured to *not* fsync() on all Append() calls
I20260810 11:30:50.014827 13611 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f300c114e8b047f2a36a606ddf2c51a4: No bootstrap required, opened a new log
I20260810 11:30:50.023054 13611 raft_consensus.cc:348] T 00000000000000000000000000000000 P f300c114e8b047f2a36a606ddf2c51a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f300c114e8b047f2a36a606ddf2c51a4" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 43199 } }
I20260810 11:30:50.023514 13611 raft_consensus.cc:374] T 00000000000000000000000000000000 P f300c114e8b047f2a36a606ddf2c51a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:30:50.023658 13611 raft_consensus.cc:729] T 00000000000000000000000000000000 P f300c114e8b047f2a36a606ddf2c51a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f300c114e8b047f2a36a606ddf2c51a4, State: Initialized, Role: FOLLOWER
I20260810 11:30:50.024320 13611 consensus_queue.cc:260] T 00000000000000000000000000000000 P f300c114e8b047f2a36a606ddf2c51a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f300c114e8b047f2a36a606ddf2c51a4" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 43199 } }
I20260810 11:30:50.024549 13611 raft_consensus.cc:388] T 00000000000000000000000000000000 P f300c114e8b047f2a36a606ddf2c51a4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 11:30:50.024711 13611 raft_consensus.cc:482] T 00000000000000000000000000000000 P f300c114e8b047f2a36a606ddf2c51a4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 11:30:50.024915 13611 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f300c114e8b047f2a36a606ddf2c51a4 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:30:50.027459 13611 raft_consensus.cc:504] T 00000000000000000000000000000000 P f300c114e8b047f2a36a606ddf2c51a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f300c114e8b047f2a36a606ddf2c51a4" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 43199 } }
I20260810 11:30:50.028213 13611 leader_election.cc:302] T 00000000000000000000000000000000 P f300c114e8b047f2a36a606ddf2c51a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f300c114e8b047f2a36a606ddf2c51a4; no voters:
I20260810 11:30:50.028906 13611 leader_election.cc:288] T 00000000000000000000000000000000 P f300c114e8b047f2a36a606ddf2c51a4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 11:30:50.029162 13616 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f300c114e8b047f2a36a606ddf2c51a4 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:30:50.029992 13616 raft_consensus.cc:686] T 00000000000000000000000000000000 P f300c114e8b047f2a36a606ddf2c51a4 [term 1 LEADER]: Becoming Leader. State: Replica: f300c114e8b047f2a36a606ddf2c51a4, State: Running, Role: LEADER
I20260810 11:30:50.030601 13616 consensus_queue.cc:237] T 00000000000000000000000000000000 P f300c114e8b047f2a36a606ddf2c51a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f300c114e8b047f2a36a606ddf2c51a4" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 43199 } }
I20260810 11:30:50.031257 13611 sys_catalog.cc:564] T 00000000000000000000000000000000 P f300c114e8b047f2a36a606ddf2c51a4 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 11:30:50.033951 13618 sys_catalog.cc:455] T 00000000000000000000000000000000 P f300c114e8b047f2a36a606ddf2c51a4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f300c114e8b047f2a36a606ddf2c51a4. Latest consensus state: current_term: 1 leader_uuid: "f300c114e8b047f2a36a606ddf2c51a4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f300c114e8b047f2a36a606ddf2c51a4" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 43199 } } }
I20260810 11:30:50.035620 13618 sys_catalog.cc:458] T 00000000000000000000000000000000 P f300c114e8b047f2a36a606ddf2c51a4 [sys.catalog]: This master's current role is: LEADER
I20260810 11:30:50.035964 13617 sys_catalog.cc:455] T 00000000000000000000000000000000 P f300c114e8b047f2a36a606ddf2c51a4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f300c114e8b047f2a36a606ddf2c51a4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f300c114e8b047f2a36a606ddf2c51a4" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 43199 } } }
I20260810 11:30:50.036872 13617 sys_catalog.cc:458] T 00000000000000000000000000000000 P f300c114e8b047f2a36a606ddf2c51a4 [sys.catalog]: This master's current role is: LEADER
I20260810 11:30:50.044077 13626 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 11:30:50.049690 13626 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 11:30:50.060359 13626 catalog_manager.cc:1347] Generated new cluster ID: 8a7f7d35e1264ac0b9d0dad2f01d947b
I20260810 11:30:50.060520 13626 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 11:30:50.080510 13626 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 11:30:50.082082 13626 catalog_manager.cc:1504] Loading token signing keys...
I20260810 11:30:50.099658 13626 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f300c114e8b047f2a36a606ddf2c51a4: Generated new TSK 0
I20260810 11:30:50.100651 13626 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260810 11:30:50.150540 13643 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260810 11:30:50.150897 13643 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260810 11:30:50.151957 13614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:50.126676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42016 (local address 127.8.18.254:43199)
0810 11:30:50.126995 (+ 319us) server_negotiation.cc:206] Beginning negotiation
0810 11:30:50.127003 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:30:50.127603 (+ 600us) server_negotiation.cc:407] Connection header received
0810 11:30:50.128090 (+ 487us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:30:50.128095 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:30:50.128295 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:30:50.128518 (+ 223us) server_negotiation.cc:226] Negotiated authn=JWT
0810 11:30:50.129508 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:50.130376 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:50.132952 (+ 2576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:50.133611 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:50.133780 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:50.134667 (+ 887us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0810 11:30:50.151346 (+ 16679us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33725/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0810 11:30:50.151694 (+ 348us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:42016: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33725/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":97}
W20260810 11:30:50.152155 13614 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:42016: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33725/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260810 11:30:50.154738 13642 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:30:50.126456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:43199 (local address 127.0.0.1:42016)
0810 11:30:50.127396 (+ 940us) negotiation.cc:107] Waiting for socket to connect
0810 11:30:50.127430 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0810 11:30:50.127693 (+ 263us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:30:50.128647 (+ 954us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:30:50.128654 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:30:50.128704 (+ 50us) client_negotiation.cc:189] Negotiated authn=JWT
0810 11:30:50.129252 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:30:50.129268 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:30:50.130562 (+ 1294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:30:50.130569 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:30:50.132722 (+ 2153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:30:50.132756 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:30:50.133891 (+ 1135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:30:50.133900 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:30:50.134158 (+ 258us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:50.134280 (+ 122us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0810 11:30:50.151757 (+ 17477us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33725/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0810 11:30:50.152210 (+ 453us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.8.18.254:43199: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33725/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":680,"thread_start_us":471,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.8.18.254:43199: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33725/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260810 11:30:50.157071 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 13543
I20260810 11:30:50.158569 13606 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:30:50.259438 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 13543
2026-08-10T11:30:50Z chronyd exiting
I20260810 11:30:50.313158 8267 test_util.cc:175] -----------------------------------------------
I20260810 11:30:50.313386 8267 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786361401922362-8267-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (847 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260810 11:30:50.317574 8267 mini_oidc.cc:152] Starting JWKS server
I20260810 11:30:50.319638 8267 webserver.cc:460] Webserver started at https://127.0.0.1:44883/ using document root <none> and password file <none>
I20260810 11:30:50.319944 8267 mini_oidc.cc:189] Starting OIDC Discovery server
I20260810 11:30:50.320703 8267 webserver.cc:460] Webserver started at http://127.0.0.1:46757/ using document root <none> and password file <none>
2026-08-10T11:30:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T11:30:50Z Disabled control of system clock
I20260810 11:30:50.356405 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786361401922362-8267-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786361401922362-8267-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786361401922362-8267-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.18.254:33811
--webserver_interface=127.8.18.254
--webserver_port=0
--builtin_ntp_servers=127.8.18.212:34407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.18.254:33811
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44883/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786361401922362-8267-0/testchainca.cert with env {}
W20260810 11:30:50.715492 13650 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:30:50.715839 13650 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:30:50.716001 13650 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:30:50.724118 13650 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 11:30:50.724264 13650 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:30:50.724335 13650 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:30:50.724444 13650 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 11:30:50.724553 13650 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260810 11:30:50.724671 13650 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260810 11:30:50.724731 13650 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:44883/jwks.json
I20260810 11:30:50.735957 13650 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.18.212:34407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786361401922362-8267-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786361401922362-8267-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.18.254:33811
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.18.254:33811
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44883/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.18.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786361401922362-8267-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786361401922362-8267-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:30:50.737594 13650 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:30:50.739100 13650 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:30:50.753352 13658 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:30:50.753638 13655 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:30:50.753388 13656 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:30:50.754359 13650 server_base.cc:1034] running on GCE node
I20260810 11:30:50.755424 13650 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:30:50.757301 13650 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:30:50.758594 13650 hybrid_clock.cc:648] HybridClock initialized: now 1786361450758540 us; error 83 us; skew 500 ppm
I20260810 11:30:50.759066 13650 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 11:30:50.762315 13650 webserver.cc:460] Webserver started at http://127.8.18.254:37097/ using document root <none> and password file <none>
I20260810 11:30:50.763258 13650 fs_manager.cc:359] Metadata directory not provided
I20260810 11:30:50.763386 13650 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:30:50.763841 13650 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:30:50.767587 13650 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786361401922362-8267-0/minicluster-data/master-0/data/instance:
uuid: "8a4665b36a0d4cc4b7ca5b7b40f59993"
format_stamp: "Formatted at 2026-08-10 11:30:50 on dist-test-slave-srgv"
server_key: "44195a284c0a081c13652fa6cae893ec"
server_key_iv: "45539329e9dab4d8729f6be741faf090"
server_key_version: "encryptionkey@0"
I20260810 11:30:50.768641 13650 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786361401922362-8267-0/minicluster-data/master-0/wal/instance:
uuid: "8a4665b36a0d4cc4b7ca5b7b40f59993"
format_stamp: "Formatted at 2026-08-10 11:30:50 on dist-test-slave-srgv"
server_key: "44195a284c0a081c13652fa6cae893ec"
server_key_iv: "45539329e9dab4d8729f6be741faf090"
server_key_version: "encryptionkey@0"
I20260810 11:30:50.775153 13650 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260810 11:30:50.779485 13664 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:30:50.781611 13650 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260810 11:30:50.781875 13650 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786361401922362-8267-0/minicluster-data/master-0/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786361401922362-8267-0/minicluster-data/master-0/wal
uuid: "8a4665b36a0d4cc4b7ca5b7b40f59993"
format_stamp: "Formatted at 2026-08-10 11:30:50 on dist-test-slave-srgv"
server_key: "44195a284c0a081c13652fa6cae893ec"
server_key_iv: "45539329e9dab4d8729f6be741faf090"
server_key_version: "encryptionkey@0"
I20260810 11:30:50.782207 13650 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786361401922362-8267-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786361401922362-8267-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786361401922362-8267-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:30:50.802163 13650 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:30:50.803188 13650 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:30:50.803565 13650 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:30:50.825527 13650 rpc_server.cc:307] RPC server started. Bound to: 127.8.18.254:33811
I20260810 11:30:50.825598 13718 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.18.254:33811 every 8 connection(s)
I20260810 11:30:50.827153 13650 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
I20260810 11:30:50.831617 13719 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:30:50.833060 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 13650
I20260810 11:30:50.833330 8267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786361401922362-8267-0/minicluster-data/master-0/wal/instance
I20260810 11:30:50.834228 8267 external_mini_cluster.cc:1428] Setting key 6e33700266202236394f058ce0c2b9c6
I20260810 11:30:50.848366 13719 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a4665b36a0d4cc4b7ca5b7b40f59993: Bootstrap starting.
I20260810 11:30:50.850634 13722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:50.836453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40494 (local address 127.8.18.254:33811)
0810 11:30:50.837903 (+ 1450us) server_negotiation.cc:206] Beginning negotiation
0810 11:30:50.837918 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 11:30:50.837977 (+ 59us) server_negotiation.cc:407] Connection header received
0810 11:30:50.839732 (+ 1755us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:30:50.839760 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:30:50.840357 (+ 597us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:30:50.840782 (+ 425us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:30:50.841469 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:50.842846 (+ 1377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:50.844960 (+ 2114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:50.845978 (+ 1018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:50.846180 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:50.847690 (+ 1510us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:30:50.847700 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:30:50.847718 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:30:50.848345 (+ 627us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:30:50.848448 (+ 103us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:30:50.848522 (+ 74us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:30:50.848649 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0810 11:30:50.849011 (+ 362us) server_negotiation.cc:299] Negotiation successful
0810 11:30:50.849470 (+ 459us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":902,"thread_start_us":455,"threads_started":1}
I20260810 11:30:50.852674 13719 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8a4665b36a0d4cc4b7ca5b7b40f59993: Neither blocks nor log segments found. Creating new log.
I20260810 11:30:50.854234 13719 log.cc:824] T 00000000000000000000000000000000 P 8a4665b36a0d4cc4b7ca5b7b40f59993: Log is configured to *not* fsync() on all Append() calls
I20260810 11:30:50.856122 8267 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 11:30:50.858091 13719 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a4665b36a0d4cc4b7ca5b7b40f59993: No bootstrap required, opened a new log
I20260810 11:30:50.866822 13719 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8a4665b36a0d4cc4b7ca5b7b40f59993 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a4665b36a0d4cc4b7ca5b7b40f59993" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 33811 } }
I20260810 11:30:50.867383 13719 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8a4665b36a0d4cc4b7ca5b7b40f59993 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:30:50.867520 13719 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8a4665b36a0d4cc4b7ca5b7b40f59993 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a4665b36a0d4cc4b7ca5b7b40f59993, State: Initialized, Role: FOLLOWER
I20260810 11:30:50.868170 13719 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8a4665b36a0d4cc4b7ca5b7b40f59993 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a4665b36a0d4cc4b7ca5b7b40f59993" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 33811 } }
I20260810 11:30:50.868427 13719 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8a4665b36a0d4cc4b7ca5b7b40f59993 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 11:30:50.868525 13719 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8a4665b36a0d4cc4b7ca5b7b40f59993 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 11:30:50.868664 13719 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8a4665b36a0d4cc4b7ca5b7b40f59993 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:30:50.871155 13719 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8a4665b36a0d4cc4b7ca5b7b40f59993 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a4665b36a0d4cc4b7ca5b7b40f59993" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 33811 } }
I20260810 11:30:50.871728 13719 leader_election.cc:302] T 00000000000000000000000000000000 P 8a4665b36a0d4cc4b7ca5b7b40f59993 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8a4665b36a0d4cc4b7ca5b7b40f59993; no voters:
I20260810 11:30:50.872354 13719 leader_election.cc:288] T 00000000000000000000000000000000 P 8a4665b36a0d4cc4b7ca5b7b40f59993 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 11:30:50.872552 13724 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8a4665b36a0d4cc4b7ca5b7b40f59993 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:30:50.873502 13724 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8a4665b36a0d4cc4b7ca5b7b40f59993 [term 1 LEADER]: Becoming Leader. State: Replica: 8a4665b36a0d4cc4b7ca5b7b40f59993, State: Running, Role: LEADER
I20260810 11:30:50.874212 13724 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8a4665b36a0d4cc4b7ca5b7b40f59993 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a4665b36a0d4cc4b7ca5b7b40f59993" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 33811 } }
I20260810 11:30:50.875036 13719 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8a4665b36a0d4cc4b7ca5b7b40f59993 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 11:30:50.877708 13725 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a4665b36a0d4cc4b7ca5b7b40f59993 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8a4665b36a0d4cc4b7ca5b7b40f59993" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a4665b36a0d4cc4b7ca5b7b40f59993" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 33811 } } }
I20260810 11:30:50.877736 13726 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a4665b36a0d4cc4b7ca5b7b40f59993 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8a4665b36a0d4cc4b7ca5b7b40f59993. Latest consensus state: current_term: 1 leader_uuid: "8a4665b36a0d4cc4b7ca5b7b40f59993" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a4665b36a0d4cc4b7ca5b7b40f59993" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 33811 } } }
I20260810 11:30:50.878305 13726 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a4665b36a0d4cc4b7ca5b7b40f59993 [sys.catalog]: This master's current role is: LEADER
I20260810 11:30:50.878305 13725 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a4665b36a0d4cc4b7ca5b7b40f59993 [sys.catalog]: This master's current role is: LEADER
I20260810 11:30:50.882277 13732 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 11:30:50.888933 13732 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 11:30:50.899564 13732 catalog_manager.cc:1347] Generated new cluster ID: 27efc87f37d145af9e5a8b30fa61b8b7
I20260810 11:30:50.899713 13732 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 11:30:50.925666 13732 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 11:30:50.927773 13732 catalog_manager.cc:1504] Loading token signing keys...
I20260810 11:30:50.942847 13732 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8a4665b36a0d4cc4b7ca5b7b40f59993: Generated new TSK 0
I20260810 11:30:50.944132 13732 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260810 11:30:50.990351 13751 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260810 11:30:50.990517 13751 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260810 11:30:50.991842 13722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:50.969179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40498 (local address 127.8.18.254:33811)
0810 11:30:50.969475 (+ 296us) server_negotiation.cc:206] Beginning negotiation
0810 11:30:50.969487 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 11:30:50.970316 (+ 829us) server_negotiation.cc:407] Connection header received
0810 11:30:50.970492 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:30:50.970496 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:30:50.970576 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:30:50.970733 (+ 157us) server_negotiation.cc:226] Negotiated authn=JWT
0810 11:30:50.971349 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:50.972114 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:50.974518 (+ 2404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:50.975152 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:50.975279 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:50.976164 (+ 885us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0810 11:30:50.991209 (+ 15045us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44883/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0810 11:30:50.991572 (+ 363us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:40498: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44883/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":84}
W20260810 11:30:50.992017 13722 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:40498: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44883/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260810 11:30:50.992300 13750 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:30:50.969121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:33811 (local address 127.0.0.1:40498)
0810 11:30:50.969929 (+ 808us) negotiation.cc:107] Waiting for socket to connect
0810 11:30:50.969954 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0810 11:30:50.970149 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:30:50.970795 (+ 646us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:30:50.970801 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:30:50.970818 (+ 17us) client_negotiation.cc:189] Negotiated authn=JWT
0810 11:30:50.971165 (+ 347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:30:50.971178 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:30:50.972284 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:30:50.972293 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:30:50.974269 (+ 1976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:30:50.974293 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:30:50.975394 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:30:50.975402 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:30:50.975633 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:50.975689 (+ 56us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0810 11:30:50.991512 (+ 15823us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44883/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0810 11:30:50.991849 (+ 337us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.8.18.254:33811: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44883/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":650,"thread_start_us":447,"threads_started":1}
I20260810 11:30:50.994102 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 13650
I20260810 11:30:50.995558 13714 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:30:51.095738 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 13650
2026-08-10T11:30:51Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (836 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260810 11:30:51.154155 8267 mini_oidc.cc:152] Starting JWKS server
I20260810 11:30:51.156316 8267 webserver.cc:460] Webserver started at https://127.0.0.1:36539/ using document root <none> and password file <none>
I20260810 11:30:51.156608 8267 mini_oidc.cc:189] Starting OIDC Discovery server
I20260810 11:30:51.157385 8267 webserver.cc:460] Webserver started at http://127.0.0.1:43453/ using document root <none> and password file <none>
2026-08-10T11:30:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T11:30:51Z Disabled control of system clock
I20260810 11:30:51.193612 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786361401922362-8267-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786361401922362-8267-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786361401922362-8267-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.18.254:45149
--webserver_interface=127.8.18.254
--webserver_port=0
--builtin_ntp_servers=127.8.18.212:45177
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.18.254:45149
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36539/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260810 11:30:51.564472 13759 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:30:51.564821 13759 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:30:51.564991 13759 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:30:51.572793 13759 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 11:30:51.572937 13759 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:30:51.573016 13759 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:30:51.573113 13759 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 11:30:51.573179 13759 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260810 11:30:51.573303 13759 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260810 11:30:51.573378 13759 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:36539/jwks.json
I20260810 11:30:51.583433 13759 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.18.212:45177
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786361401922362-8267-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786361401922362-8267-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.18.254:45149
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.18.254:45149
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36539/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.18.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786361401922362-8267-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:30:51.584965 13759 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:30:51.587011 13759 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:30:51.600662 13764 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:30:51.600703 13765 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:30:51.600744 13767 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:30:51.601653 13759 server_base.cc:1034] running on GCE node
I20260810 11:30:51.602559 13759 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:30:51.604180 13759 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:30:51.605513 13759 hybrid_clock.cc:648] HybridClock initialized: now 1786361451605449 us; error 65 us; skew 500 ppm
I20260810 11:30:51.605937 13759 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 11:30:51.612742 13759 webserver.cc:460] Webserver started at http://127.8.18.254:44327/ using document root <none> and password file <none>
I20260810 11:30:51.613757 13759 fs_manager.cc:359] Metadata directory not provided
I20260810 11:30:51.613920 13759 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:30:51.614310 13759 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:30:51.617789 13759 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786361401922362-8267-0/minicluster-data/master-0/data/instance:
uuid: "b500df37e9ef4c7b83f2d63a14bb6f70"
format_stamp: "Formatted at 2026-08-10 11:30:51 on dist-test-slave-srgv"
server_key: "5e73d1a821ab21898cc5b1ebac535ca2"
server_key_iv: "afb1153e09732b9a77e6edc87856fc3f"
server_key_version: "encryptionkey@0"
I20260810 11:30:51.618760 13759 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786361401922362-8267-0/minicluster-data/master-0/wal/instance:
uuid: "b500df37e9ef4c7b83f2d63a14bb6f70"
format_stamp: "Formatted at 2026-08-10 11:30:51 on dist-test-slave-srgv"
server_key: "5e73d1a821ab21898cc5b1ebac535ca2"
server_key_iv: "afb1153e09732b9a77e6edc87856fc3f"
server_key_version: "encryptionkey@0"
I20260810 11:30:51.624961 13759 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260810 11:30:51.629307 13774 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:30:51.631158 13759 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.003s
I20260810 11:30:51.631470 13759 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786361401922362-8267-0/minicluster-data/master-0/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786361401922362-8267-0/minicluster-data/master-0/wal
uuid: "b500df37e9ef4c7b83f2d63a14bb6f70"
format_stamp: "Formatted at 2026-08-10 11:30:51 on dist-test-slave-srgv"
server_key: "5e73d1a821ab21898cc5b1ebac535ca2"
server_key_iv: "afb1153e09732b9a77e6edc87856fc3f"
server_key_version: "encryptionkey@0"
I20260810 11:30:51.631778 13759 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786361401922362-8267-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786361401922362-8267-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786361401922362-8267-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:30:51.657003 13759 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:30:51.658054 13759 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:30:51.658437 13759 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:30:51.679961 13759 rpc_server.cc:307] RPC server started. Bound to: 127.8.18.254:45149
I20260810 11:30:51.679991 13826 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.18.254:45149 every 8 connection(s)
I20260810 11:30:51.681630 13759 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
I20260810 11:30:51.682585 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 13759
I20260810 11:30:51.682873 8267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786361401922362-8267-0/minicluster-data/master-0/wal/instance
I20260810 11:30:51.683665 8267 external_mini_cluster.cc:1428] Setting key 7459fb820b810ba3a6ef9bc186797688
I20260810 11:30:51.686887 13827 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:30:51.701849 13830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:51.685922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40670 (local address 127.8.18.254:45149)
0810 11:30:51.687117 (+ 1195us) server_negotiation.cc:206] Beginning negotiation
0810 11:30:51.687134 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0810 11:30:51.687193 (+ 59us) server_negotiation.cc:407] Connection header received
0810 11:30:51.689714 (+ 2521us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:30:51.689747 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:30:51.690498 (+ 751us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:30:51.691051 (+ 553us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:30:51.691735 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:51.693914 (+ 2179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:51.695541 (+ 1627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:51.696688 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:51.696913 (+ 225us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:51.698467 (+ 1554us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:30:51.698480 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:30:51.698508 (+ 28us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:30:51.699354 (+ 846us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:30:51.699473 (+ 119us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:30:51.699504 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:30:51.699638 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0810 11:30:51.699938 (+ 300us) server_negotiation.cc:299] Negotiation successful
0810 11:30:51.700367 (+ 429us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":630,"thread_start_us":380,"threads_started":1}
I20260810 11:30:51.703073 13827 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b500df37e9ef4c7b83f2d63a14bb6f70: Bootstrap starting.
I20260810 11:30:51.707020 8267 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 11:30:51.707610 13827 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b500df37e9ef4c7b83f2d63a14bb6f70: Neither blocks nor log segments found. Creating new log.
I20260810 11:30:51.709079 13827 log.cc:824] T 00000000000000000000000000000000 P b500df37e9ef4c7b83f2d63a14bb6f70: Log is configured to *not* fsync() on all Append() calls
I20260810 11:30:51.712826 13827 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b500df37e9ef4c7b83f2d63a14bb6f70: No bootstrap required, opened a new log
I20260810 11:30:51.721910 13827 raft_consensus.cc:348] T 00000000000000000000000000000000 P b500df37e9ef4c7b83f2d63a14bb6f70 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b500df37e9ef4c7b83f2d63a14bb6f70" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 45149 } }
I20260810 11:30:51.722461 13827 raft_consensus.cc:374] T 00000000000000000000000000000000 P b500df37e9ef4c7b83f2d63a14bb6f70 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:30:51.722586 13827 raft_consensus.cc:729] T 00000000000000000000000000000000 P b500df37e9ef4c7b83f2d63a14bb6f70 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b500df37e9ef4c7b83f2d63a14bb6f70, State: Initialized, Role: FOLLOWER
I20260810 11:30:51.723174 13827 consensus_queue.cc:260] T 00000000000000000000000000000000 P b500df37e9ef4c7b83f2d63a14bb6f70 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b500df37e9ef4c7b83f2d63a14bb6f70" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 45149 } }
I20260810 11:30:51.723403 13827 raft_consensus.cc:388] T 00000000000000000000000000000000 P b500df37e9ef4c7b83f2d63a14bb6f70 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 11:30:51.723515 13827 raft_consensus.cc:482] T 00000000000000000000000000000000 P b500df37e9ef4c7b83f2d63a14bb6f70 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 11:30:51.723658 13827 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b500df37e9ef4c7b83f2d63a14bb6f70 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:30:51.726197 13827 raft_consensus.cc:504] T 00000000000000000000000000000000 P b500df37e9ef4c7b83f2d63a14bb6f70 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b500df37e9ef4c7b83f2d63a14bb6f70" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 45149 } }
I20260810 11:30:51.726855 13827 leader_election.cc:302] T 00000000000000000000000000000000 P b500df37e9ef4c7b83f2d63a14bb6f70 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b500df37e9ef4c7b83f2d63a14bb6f70; no voters:
I20260810 11:30:51.727543 13827 leader_election.cc:288] T 00000000000000000000000000000000 P b500df37e9ef4c7b83f2d63a14bb6f70 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 11:30:51.727859 13832 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b500df37e9ef4c7b83f2d63a14bb6f70 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:30:51.729266 13832 raft_consensus.cc:686] T 00000000000000000000000000000000 P b500df37e9ef4c7b83f2d63a14bb6f70 [term 1 LEADER]: Becoming Leader. State: Replica: b500df37e9ef4c7b83f2d63a14bb6f70, State: Running, Role: LEADER
I20260810 11:30:51.730043 13832 consensus_queue.cc:237] T 00000000000000000000000000000000 P b500df37e9ef4c7b83f2d63a14bb6f70 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b500df37e9ef4c7b83f2d63a14bb6f70" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 45149 } }
I20260810 11:30:51.730455 13827 sys_catalog.cc:564] T 00000000000000000000000000000000 P b500df37e9ef4c7b83f2d63a14bb6f70 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 11:30:51.734586 13834 sys_catalog.cc:455] T 00000000000000000000000000000000 P b500df37e9ef4c7b83f2d63a14bb6f70 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b500df37e9ef4c7b83f2d63a14bb6f70. Latest consensus state: current_term: 1 leader_uuid: "b500df37e9ef4c7b83f2d63a14bb6f70" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b500df37e9ef4c7b83f2d63a14bb6f70" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 45149 } } }
I20260810 11:30:51.734601 13833 sys_catalog.cc:455] T 00000000000000000000000000000000 P b500df37e9ef4c7b83f2d63a14bb6f70 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b500df37e9ef4c7b83f2d63a14bb6f70" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b500df37e9ef4c7b83f2d63a14bb6f70" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 45149 } } }
I20260810 11:30:51.735165 13833 sys_catalog.cc:458] T 00000000000000000000000000000000 P b500df37e9ef4c7b83f2d63a14bb6f70 [sys.catalog]: This master's current role is: LEADER
I20260810 11:30:51.735350 13834 sys_catalog.cc:458] T 00000000000000000000000000000000 P b500df37e9ef4c7b83f2d63a14bb6f70 [sys.catalog]: This master's current role is: LEADER
I20260810 11:30:51.742691 13841 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 11:30:51.749198 13841 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 11:30:51.759183 13841 catalog_manager.cc:1347] Generated new cluster ID: 574d28f0fdbd479c962160afde9747d4
I20260810 11:30:51.759341 13841 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 11:30:51.771340 13841 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 11:30:51.773583 13841 catalog_manager.cc:1504] Loading token signing keys...
I20260810 11:30:51.783730 13841 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b500df37e9ef4c7b83f2d63a14bb6f70: Generated new TSK 0
I20260810 11:30:51.784978 13841 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 11:30:51.822779 13830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:51.812710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40674 (local address 127.8.18.254:45149)
0810 11:30:51.812935 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0810 11:30:51.812942 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:30:51.813375 (+ 433us) server_negotiation.cc:407] Connection header received
0810 11:30:51.813721 (+ 346us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:30:51.813729 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:30:51.813847 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:30:51.814059 (+ 212us) server_negotiation.cc:226] Negotiated authn=JWT
0810 11:30:51.814574 (+ 515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:51.815799 (+ 1225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:51.817566 (+ 1767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:51.818331 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:51.818503 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:51.819278 (+ 775us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0810 11:30:51.822009 (+ 2731us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36539/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0810 11:30:51.822437 (+ 428us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:40674: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36539/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":84}
W20260810 11:30:51.822940 13830 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:40674: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36539/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260810 11:30:51.824635 13857 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:30:51.812519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:45149 (local address 127.0.0.1:40674)
0810 11:30:51.813248 (+ 729us) negotiation.cc:107] Waiting for socket to connect
0810 11:30:51.813267 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 11:30:51.813474 (+ 207us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:30:51.814103 (+ 629us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:30:51.814107 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:30:51.814118 (+ 11us) client_negotiation.cc:189] Negotiated authn=JWT
0810 11:30:51.814397 (+ 279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:30:51.814406 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:30:51.816085 (+ 1679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:30:51.816090 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:30:51.817284 (+ 1194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:30:51.817302 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:30:51.818659 (+ 1357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:30:51.818666 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:30:51.818866 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:51.818911 (+ 45us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0810 11:30:51.822314 (+ 3403us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36539/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0810 11:30:51.823951 (+ 1637us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.8.18.254:45149: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36539/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":562,"mutex_wait_us":1235,"thread_start_us":409,"threads_started":1}
I20260810 11:30:51.826040 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 13759
I20260810 11:30:51.827492 13822 generic_service.cc:196] Checking for leaks (request via RPC)
E20260810 11:30:51.837857 13858 webserver.cc:533] Webserver: sslize failed:
E20260810 11:30:51.838032 13858 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260810 11:30:51.929091 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 13759
2026-08-10T11:30:51Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (835 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-10T11:30:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T11:30:52Z Disabled control of system clock
I20260810 11:30:52.024585 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786361401922362-8267-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786361401922362-8267-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786361401922362-8267-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.18.254:44217
--webserver_interface=127.8.18.254
--webserver_port=0
--builtin_ntp_servers=127.8.18.212:41725
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.18.254:44217
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786361401922362-8267-0/testchainca.cert with env {}
W20260810 11:30:52.384814 13863 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:30:52.385200 13863 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:30:52.385367 13863 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:30:52.393441 13863 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 11:30:52.393585 13863 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:30:52.393662 13863 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:30:52.393757 13863 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 11:30:52.393836 13863 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260810 11:30:52.393931 13863 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260810 11:30:52.394003 13863 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260810 11:30:52.405007 13863 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.18.212:41725
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786361401922362-8267-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786361401922362-8267-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.18.254:44217
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.18.254:44217
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.18.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786361401922362-8267-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786361401922362-8267-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:30:52.406635 13863 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:30:52.408169 13863 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:30:52.422456 13868 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:30:52.422318 13869 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:30:52.422411 13872 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:30:52.423529 13863 server_base.cc:1034] running on GCE node
I20260810 11:30:52.424532 13863 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:30:52.426383 13863 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:30:52.427757 13863 hybrid_clock.cc:648] HybridClock initialized: now 1786361452427661 us; error 68 us; skew 500 ppm
I20260810 11:30:52.428220 13863 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 11:30:52.431114 13863 webserver.cc:460] Webserver started at http://127.8.18.254:33585/ using document root <none> and password file <none>
I20260810 11:30:52.431968 13863 fs_manager.cc:359] Metadata directory not provided
I20260810 11:30:52.432130 13863 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:30:52.432495 13863 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:30:52.435873 13863 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786361401922362-8267-0/minicluster-data/master-0/data/instance:
uuid: "1ec7ee9ff8934f27a3529291e873cfda"
format_stamp: "Formatted at 2026-08-10 11:30:52 on dist-test-slave-srgv"
server_key: "12058a7b7ec54ffa987d5c8903122812"
server_key_iv: "221978881a584262a28a1ccdc33be09f"
server_key_version: "encryptionkey@0"
I20260810 11:30:52.436894 13863 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786361401922362-8267-0/minicluster-data/master-0/wal/instance:
uuid: "1ec7ee9ff8934f27a3529291e873cfda"
format_stamp: "Formatted at 2026-08-10 11:30:52 on dist-test-slave-srgv"
server_key: "12058a7b7ec54ffa987d5c8903122812"
server_key_iv: "221978881a584262a28a1ccdc33be09f"
server_key_version: "encryptionkey@0"
I20260810 11:30:52.443437 13863 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.009s sys 0.000s
I20260810 11:30:52.447867 13878 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:30:52.449641 13863 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260810 11:30:52.449926 13863 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786361401922362-8267-0/minicluster-data/master-0/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786361401922362-8267-0/minicluster-data/master-0/wal
uuid: "1ec7ee9ff8934f27a3529291e873cfda"
format_stamp: "Formatted at 2026-08-10 11:30:52 on dist-test-slave-srgv"
server_key: "12058a7b7ec54ffa987d5c8903122812"
server_key_iv: "221978881a584262a28a1ccdc33be09f"
server_key_version: "encryptionkey@0"
I20260810 11:30:52.450201 13863 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786361401922362-8267-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786361401922362-8267-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786361401922362-8267-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:30:52.463554 13863 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:30:52.464603 13863 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:30:52.464980 13863 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:30:52.486606 13863 rpc_server.cc:307] RPC server started. Bound to: 127.8.18.254:44217
I20260810 11:30:52.486629 13930 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.18.254:44217 every 8 connection(s)
I20260810 11:30:52.488147 13863 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
I20260810 11:30:52.490960 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 13863
I20260810 11:30:52.491241 8267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786361401922362-8267-0/minicluster-data/master-0/wal/instance
I20260810 11:30:52.491990 8267 external_mini_cluster.cc:1428] Setting key 382fa05154ef65d0b25776a329380238
I20260810 11:30:52.492579 13931 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:30:52.507558 13931 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ec7ee9ff8934f27a3529291e873cfda: Bootstrap starting.
I20260810 11:30:52.509599 13934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:52.494208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44094 (local address 127.8.18.254:44217)
0810 11:30:52.495337 (+ 1129us) server_negotiation.cc:206] Beginning negotiation
0810 11:30:52.495349 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 11:30:52.495404 (+ 55us) server_negotiation.cc:407] Connection header received
0810 11:30:52.497871 (+ 2467us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:30:52.497903 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:30:52.498616 (+ 713us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:30:52.499142 (+ 526us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:30:52.499913 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:52.501606 (+ 1693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:52.503199 (+ 1593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:52.504255 (+ 1056us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:52.504783 (+ 528us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:52.506395 (+ 1612us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:30:52.506405 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:30:52.506423 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:30:52.507084 (+ 661us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:30:52.507190 (+ 106us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:30:52.507219 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:30:52.507415 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0810 11:30:52.507710 (+ 295us) server_negotiation.cc:299] Negotiation successful
0810 11:30:52.508342 (+ 632us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":645,"thread_start_us":407,"threads_started":1}
I20260810 11:30:52.511788 13931 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1ec7ee9ff8934f27a3529291e873cfda: Neither blocks nor log segments found. Creating new log.
I20260810 11:30:52.513144 13931 log.cc:824] T 00000000000000000000000000000000 P 1ec7ee9ff8934f27a3529291e873cfda: Log is configured to *not* fsync() on all Append() calls
I20260810 11:30:52.514853 8267 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 11:30:52.516973 13931 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ec7ee9ff8934f27a3529291e873cfda: No bootstrap required, opened a new log
I20260810 11:30:52.526302 13931 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1ec7ee9ff8934f27a3529291e873cfda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ec7ee9ff8934f27a3529291e873cfda" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 44217 } }
I20260810 11:30:52.526754 13931 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1ec7ee9ff8934f27a3529291e873cfda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:30:52.526849 13931 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1ec7ee9ff8934f27a3529291e873cfda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ec7ee9ff8934f27a3529291e873cfda, State: Initialized, Role: FOLLOWER
I20260810 11:30:52.527511 13931 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1ec7ee9ff8934f27a3529291e873cfda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ec7ee9ff8934f27a3529291e873cfda" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 44217 } }
I20260810 11:30:52.527768 13931 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1ec7ee9ff8934f27a3529291e873cfda [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 11:30:52.527884 13931 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1ec7ee9ff8934f27a3529291e873cfda [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 11:30:52.528021 13931 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1ec7ee9ff8934f27a3529291e873cfda [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:30:52.530656 13931 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1ec7ee9ff8934f27a3529291e873cfda [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ec7ee9ff8934f27a3529291e873cfda" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 44217 } }
I20260810 11:30:52.531301 13931 leader_election.cc:302] T 00000000000000000000000000000000 P 1ec7ee9ff8934f27a3529291e873cfda [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1ec7ee9ff8934f27a3529291e873cfda; no voters:
I20260810 11:30:52.532058 13931 leader_election.cc:288] T 00000000000000000000000000000000 P 1ec7ee9ff8934f27a3529291e873cfda [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 11:30:52.532209 13936 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1ec7ee9ff8934f27a3529291e873cfda [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:30:52.533649 13936 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1ec7ee9ff8934f27a3529291e873cfda [term 1 LEADER]: Becoming Leader. State: Replica: 1ec7ee9ff8934f27a3529291e873cfda, State: Running, Role: LEADER
I20260810 11:30:52.534264 13936 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1ec7ee9ff8934f27a3529291e873cfda [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ec7ee9ff8934f27a3529291e873cfda" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 44217 } }
I20260810 11:30:52.534799 13931 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1ec7ee9ff8934f27a3529291e873cfda [sys.catalog]: configured and running, proceeding with master startup.
I20260810 11:30:52.537985 13937 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ec7ee9ff8934f27a3529291e873cfda [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1ec7ee9ff8934f27a3529291e873cfda" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ec7ee9ff8934f27a3529291e873cfda" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 44217 } } }
I20260810 11:30:52.538540 13937 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ec7ee9ff8934f27a3529291e873cfda [sys.catalog]: This master's current role is: LEADER
I20260810 11:30:52.539177 13938 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ec7ee9ff8934f27a3529291e873cfda [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1ec7ee9ff8934f27a3529291e873cfda. Latest consensus state: current_term: 1 leader_uuid: "1ec7ee9ff8934f27a3529291e873cfda" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ec7ee9ff8934f27a3529291e873cfda" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 44217 } } }
I20260810 11:30:52.539597 13938 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ec7ee9ff8934f27a3529291e873cfda [sys.catalog]: This master's current role is: LEADER
I20260810 11:30:52.544682 13942 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 11:30:52.552500 13942 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 11:30:52.564181 13942 catalog_manager.cc:1347] Generated new cluster ID: 60e5b7d6a5454e049d94ead8dbd79a78
I20260810 11:30:52.564411 13942 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 11:30:52.591917 13942 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 11:30:52.593492 13942 catalog_manager.cc:1504] Loading token signing keys...
I20260810 11:30:52.612277 13942 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1ec7ee9ff8934f27a3529291e873cfda: Generated new TSK 0
I20260810 11:30:52.613497 13942 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 11:30:52.713014 13934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:52.641126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44096 (local address 127.8.18.254:44217)
0810 11:30:52.641360 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0810 11:30:52.641370 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 11:30:52.641994 (+ 624us) server_negotiation.cc:407] Connection header received
0810 11:30:52.642261 (+ 267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:30:52.642267 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:30:52.642369 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:30:52.642585 (+ 216us) server_negotiation.cc:226] Negotiated authn=JWT
0810 11:30:52.643182 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:52.644366 (+ 1184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:52.646116 (+ 1750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:52.646888 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:52.647032 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:52.647898 (+ 866us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0810 11:30:52.712215 (+ 64317us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0810 11:30:52.712679 (+ 464us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:44096: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":65}
W20260810 11:30:52.713199 13934 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:44096: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260810 11:30:52.713375 13961 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:30:52.641098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:44217 (local address 127.0.0.1:44096)
0810 11:30:52.641874 (+ 776us) negotiation.cc:107] Waiting for socket to connect
0810 11:30:52.641891 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 11:30:52.642041 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:30:52.642632 (+ 591us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:30:52.642636 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:30:52.642647 (+ 11us) client_negotiation.cc:189] Negotiated authn=JWT
0810 11:30:52.642956 (+ 309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:30:52.642965 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:30:52.644564 (+ 1599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:30:52.644570 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:30:52.645934 (+ 1364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:30:52.645950 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:30:52.647123 (+ 1173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:30:52.647129 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:30:52.647453 (+ 324us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:52.647562 (+ 109us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0810 11:30:52.712588 (+ 65026us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0810 11:30:52.712861 (+ 273us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.8.18.254:44217: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":618,"thread_start_us":440,"threads_started":1}
I20260810 11:30:52.715209 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 13863
I20260810 11:30:52.716593 13926 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:30:52.823242 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 13863
2026-08-10T11:30:52Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (872 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (460 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (463 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786361401922362-8267-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 11:30:53 dist-test-slave-srgv krb5kdc[13976](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 11:30:53 dist-test-slave-srgv krb5kdc[13976](info): set up 2 sockets
Aug 10 11:30:53 dist-test-slave-srgv krb5kdc[13976](info): commencing operation
krb5kdc: starting...
W20260810 11:30:54.893496 8267 mini_kdc.cc:121] Time spent starting KDC: real 1.106s user 0.003s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786361401922362-8267-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786361401922362-8267-0/krb5kdc/test-user.keytab.
Aug 10 11:30:55 dist-test-slave-srgv krb5kdc[13976](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361455, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T11:30:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T11:30:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.18.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.18.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.18.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.254.keytab.
Entry for principal kudu/127.8.18.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.254.keytab.
I20260810 11:30:55.154745 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786361401922362-8267-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786361401922362-8267-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786361401922362-8267-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.18.254:35451
--webserver_interface=127.8.18.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.254.keytab
--principal=kudu/127.8.18.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.18.212:39391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.18.254:35451
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786361401922362-8267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786361401922362-8267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786361401922362-8267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:30:55.520418 13990 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:30:55.520750 13990 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:30:55.520846 13990 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:30:55.528628 13990 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 11:30:55.528767 13990 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:30:55.528817 13990 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:30:55.528877 13990 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 11:30:55.528923 13990 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 11:30:55.539624 13990 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.18.212:39391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786361401922362-8267-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786361401922362-8267-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.18.254:35451
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.254.keytab
--principal=kudu/127.8.18.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.18.254:35451
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.18.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786361401922362-8267-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:30:55.541204 13990 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:30:55.542755 13990 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:30:55.557061 13995 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:30:55.557147 13999 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:30:55.557157 13997 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:30:55.563553 13990 server_base.cc:1034] running on GCE node
I20260810 11:30:55.564842 13990 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:30:55.566685 13990 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:30:55.567937 13990 hybrid_clock.cc:648] HybridClock initialized: now 1786361455567893 us; error 48 us; skew 500 ppm
I20260810 11:30:55.568459 13990 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:30:55 dist-test-slave-srgv krb5kdc[13976](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361455, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:30:55.572386 13990 init.cc:377] Logged in from keytab as kudu/127.8.18.254@KRBTEST.COM (short username kudu)
I20260810 11:30:55.574358 13990 webserver.cc:460] Webserver started at http://127.8.18.254:43831/ using document root <none> and password file <none>
I20260810 11:30:55.575462 13990 fs_manager.cc:359] Metadata directory not provided
I20260810 11:30:55.575626 13990 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:30:55.576290 13990 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:30:55.580346 13990 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786361401922362-8267-0/minicluster-data/master-0/data/instance:
uuid: "bf19b84f8bd3408db569acf32c4b84d4"
format_stamp: "Formatted at 2026-08-10 11:30:55 on dist-test-slave-srgv"
server_key: "fb4944caaa0d64aa659d4f78cf96fe0d"
server_key_iv: "ceed1b96941700544f64fc737c80a64a"
server_key_version: "encryptionkey@0"
I20260810 11:30:55.581399 13990 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786361401922362-8267-0/minicluster-data/master-0/wal/instance:
uuid: "bf19b84f8bd3408db569acf32c4b84d4"
format_stamp: "Formatted at 2026-08-10 11:30:55 on dist-test-slave-srgv"
server_key: "fb4944caaa0d64aa659d4f78cf96fe0d"
server_key_iv: "ceed1b96941700544f64fc737c80a64a"
server_key_version: "encryptionkey@0"
I20260810 11:30:55.588044 13990 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260810 11:30:55.592531 14006 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:30:55.594376 13990 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260810 11:30:55.594614 13990 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786361401922362-8267-0/minicluster-data/master-0/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786361401922362-8267-0/minicluster-data/master-0/wal
uuid: "bf19b84f8bd3408db569acf32c4b84d4"
format_stamp: "Formatted at 2026-08-10 11:30:55 on dist-test-slave-srgv"
server_key: "fb4944caaa0d64aa659d4f78cf96fe0d"
server_key_iv: "ceed1b96941700544f64fc737c80a64a"
server_key_version: "encryptionkey@0"
I20260810 11:30:55.594861 13990 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786361401922362-8267-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786361401922362-8267-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786361401922362-8267-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:30:55.608214 13990 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:30:55.613368 13990 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:30:55.613811 13990 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:30:55.635346 13990 rpc_server.cc:307] RPC server started. Bound to: 127.8.18.254:35451
I20260810 11:30:55.635416 14063 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.18.254:35451 every 8 connection(s)
I20260810 11:30:55.637075 13990 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
I20260810 11:30:55.641773 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 13990
I20260810 11:30:55.642056 8267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786361401922362-8267-0/minicluster-data/master-0/wal/instance
I20260810 11:30:55.642064 14064 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:30:55.642901 8267 external_mini_cluster.cc:1428] Setting key d1636ee080274e804fb76552e5bcd427
I20260810 11:30:55.658022 14064 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bf19b84f8bd3408db569acf32c4b84d4: Bootstrap starting.
Aug 10 11:30:55 dist-test-slave-srgv krb5kdc[13976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361455, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.18.254@KRBTEST.COM
I20260810 11:30:55.662246 14064 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bf19b84f8bd3408db569acf32c4b84d4: Neither blocks nor log segments found. Creating new log.
I20260810 11:30:55.663465 14064 log.cc:824] T 00000000000000000000000000000000 P bf19b84f8bd3408db569acf32c4b84d4: Log is configured to *not* fsync() on all Append() calls
I20260810 11:30:55.666886 14064 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bf19b84f8bd3408db569acf32c4b84d4: No bootstrap required, opened a new log
I20260810 11:30:55.671057 14067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:55.645493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51790 (local address 127.8.18.254:35451)
0810 11:30:55.646516 (+ 1023us) server_negotiation.cc:206] Beginning negotiation
0810 11:30:55.646529 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 11:30:55.646577 (+ 48us) server_negotiation.cc:407] Connection header received
0810 11:30:55.649022 (+ 2445us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:30:55.649055 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:30:55.649695 (+ 640us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:30:55.650147 (+ 452us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:30:55.652159 (+ 2012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:55.653692 (+ 1533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:55.655650 (+ 1958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:55.656538 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:55.656710 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:55.662091 (+ 5381us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:30:55.662101 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:30:55.662128 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:30:55.662175 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:30:55.666017 (+ 3842us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:55.666889 (+ 872us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:55.666897 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:55.666904 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:55.667175 (+ 271us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:55.667642 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:55.667647 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:55.667651 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:55.668242 (+ 591us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:30:55.668601 (+ 359us) server_negotiation.cc:1035] Waiting for connection context
0810 11:30:55.669143 (+ 542us) server_negotiation.cc:299] Negotiation successful
0810 11:30:55.669886 (+ 743us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":781,"thread_start_us":497,"threads_started":1}
I20260810 11:30:55.676399 14064 raft_consensus.cc:348] T 00000000000000000000000000000000 P bf19b84f8bd3408db569acf32c4b84d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf19b84f8bd3408db569acf32c4b84d4" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 35451 } }
I20260810 11:30:55.676892 14064 raft_consensus.cc:374] T 00000000000000000000000000000000 P bf19b84f8bd3408db569acf32c4b84d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:30:55.677019 14064 raft_consensus.cc:729] T 00000000000000000000000000000000 P bf19b84f8bd3408db569acf32c4b84d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf19b84f8bd3408db569acf32c4b84d4, State: Initialized, Role: FOLLOWER
I20260810 11:30:55.677758 14064 consensus_queue.cc:260] T 00000000000000000000000000000000 P bf19b84f8bd3408db569acf32c4b84d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf19b84f8bd3408db569acf32c4b84d4" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 35451 } }
I20260810 11:30:55.678042 14064 raft_consensus.cc:388] T 00000000000000000000000000000000 P bf19b84f8bd3408db569acf32c4b84d4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 11:30:55.678198 14064 raft_consensus.cc:482] T 00000000000000000000000000000000 P bf19b84f8bd3408db569acf32c4b84d4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 11:30:55.678426 14064 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bf19b84f8bd3408db569acf32c4b84d4 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:30:55.682111 14064 raft_consensus.cc:504] T 00000000000000000000000000000000 P bf19b84f8bd3408db569acf32c4b84d4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf19b84f8bd3408db569acf32c4b84d4" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 35451 } }
I20260810 11:30:55.682740 14064 leader_election.cc:302] T 00000000000000000000000000000000 P bf19b84f8bd3408db569acf32c4b84d4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bf19b84f8bd3408db569acf32c4b84d4; no voters:
I20260810 11:30:55.683619 14064 leader_election.cc:288] T 00000000000000000000000000000000 P bf19b84f8bd3408db569acf32c4b84d4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 11:30:55.683884 14069 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bf19b84f8bd3408db569acf32c4b84d4 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:30:55.686365 14069 raft_consensus.cc:686] T 00000000000000000000000000000000 P bf19b84f8bd3408db569acf32c4b84d4 [term 1 LEADER]: Becoming Leader. State: Replica: bf19b84f8bd3408db569acf32c4b84d4, State: Running, Role: LEADER
I20260810 11:30:55.687018 14064 sys_catalog.cc:564] T 00000000000000000000000000000000 P bf19b84f8bd3408db569acf32c4b84d4 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 11:30:55.686941 14069 consensus_queue.cc:237] T 00000000000000000000000000000000 P bf19b84f8bd3408db569acf32c4b84d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf19b84f8bd3408db569acf32c4b84d4" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 35451 } }
I20260810 11:30:55.690989 14071 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf19b84f8bd3408db569acf32c4b84d4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bf19b84f8bd3408db569acf32c4b84d4. Latest consensus state: current_term: 1 leader_uuid: "bf19b84f8bd3408db569acf32c4b84d4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf19b84f8bd3408db569acf32c4b84d4" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 35451 } } }
I20260810 11:30:55.691198 14070 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf19b84f8bd3408db569acf32c4b84d4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bf19b84f8bd3408db569acf32c4b84d4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf19b84f8bd3408db569acf32c4b84d4" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 35451 } } }
I20260810 11:30:55.691598 14071 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf19b84f8bd3408db569acf32c4b84d4 [sys.catalog]: This master's current role is: LEADER
I20260810 11:30:55.691596 14070 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf19b84f8bd3408db569acf32c4b84d4 [sys.catalog]: This master's current role is: LEADER
I20260810 11:30:55.692744 14080 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 11:30:55.699236 14080 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 11:30:55.713192 14080 catalog_manager.cc:1347] Generated new cluster ID: 94bf80c590eb4361ae1fb49374c9a226
I20260810 11:30:55.713356 14080 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 11:30:55.724581 14080 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 11:30:55.726128 14080 catalog_manager.cc:1504] Loading token signing keys...
I20260810 11:30:55.737970 14080 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bf19b84f8bd3408db569acf32c4b84d4: Generated new TSK 0
I20260810 11:30:55.739136 14080 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.18.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.18.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.18.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.193.keytab.
Entry for principal kudu/127.8.18.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.193.keytab.
I20260810 11:30:55.814762 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786361401922362-8267-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786361401922362-8267-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786361401922362-8267-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786361401922362-8267-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.18.193:0
--local_ip_for_outbound_sockets=127.8.18.193
--webserver_interface=127.8.18.193
--webserver_port=0
--tserver_master_addrs=127.8.18.254:35451
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.193.keytab
--principal=kudu/127.8.18.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.18.212:39391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786361401922362-8267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786361401922362-8267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786361401922362-8267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:30:56.182938 14093 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:30:56.183269 14093 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:30:56.183369 14093 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:30:56.191291 14093 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:30:56.191434 14093 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:30:56.191555 14093 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.18.193
I20260810 11:30:56.203346 14093 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.18.212:39391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786361401922362-8267-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786361401922362-8267-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.193.keytab
--principal=kudu/127.8.18.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.18.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786361401922362-8267-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.18.193
--webserver_port=0
--tserver_master_addrs=127.8.18.254:35451
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.18.193
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786361401922362-8267-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:30:56.205643 14093 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:30:56.207681 14093 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:30:56.223531 14099 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:30:56.223727 14098 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:30:56.223506 14101 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:30:56.224772 14093 server_base.cc:1034] running on GCE node
I20260810 11:30:56.225558 14093 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:30:56.226655 14093 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:30:56.227921 14093 hybrid_clock.cc:648] HybridClock initialized: now 1786361456227874 us; error 49 us; skew 500 ppm
I20260810 11:30:56.228281 14093 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:30:56 dist-test-slave-srgv krb5kdc[13976](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361456, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:30:56.231906 14093 init.cc:377] Logged in from keytab as kudu/127.8.18.193@KRBTEST.COM (short username kudu)
I20260810 11:30:56.233786 14093 webserver.cc:460] Webserver started at http://127.8.18.193:33535/ using document root <none> and password file <none>
I20260810 11:30:56.234579 14093 fs_manager.cc:359] Metadata directory not provided
I20260810 11:30:56.234757 14093 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:30:56.235116 14093 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:30:56.238704 14093 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786361401922362-8267-0/minicluster-data/ts-0/data/instance:
uuid: "ac22ecce71aa4b2980b3ff051fd0b344"
format_stamp: "Formatted at 2026-08-10 11:30:56 on dist-test-slave-srgv"
server_key: "60dd0b1faaaaff740d44af1442cdf5b0"
server_key_iv: "2d503a45fdb8da63878c2474d42bdb97"
server_key_version: "encryptionkey@0"
I20260810 11:30:56.239774 14093 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786361401922362-8267-0/minicluster-data/ts-0/wal/instance:
uuid: "ac22ecce71aa4b2980b3ff051fd0b344"
format_stamp: "Formatted at 2026-08-10 11:30:56 on dist-test-slave-srgv"
server_key: "60dd0b1faaaaff740d44af1442cdf5b0"
server_key_iv: "2d503a45fdb8da63878c2474d42bdb97"
server_key_version: "encryptionkey@0"
I20260810 11:30:56.246379 14093 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.005s
I20260810 11:30:56.250658 14109 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:30:56.252332 14093 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260810 11:30:56.252571 14093 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786361401922362-8267-0/minicluster-data/ts-0/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786361401922362-8267-0/minicluster-data/ts-0/wal
uuid: "ac22ecce71aa4b2980b3ff051fd0b344"
format_stamp: "Formatted at 2026-08-10 11:30:56 on dist-test-slave-srgv"
server_key: "60dd0b1faaaaff740d44af1442cdf5b0"
server_key_iv: "2d503a45fdb8da63878c2474d42bdb97"
server_key_version: "encryptionkey@0"
I20260810 11:30:56.252833 14093 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786361401922362-8267-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786361401922362-8267-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786361401922362-8267-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:30:56.287226 14093 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:30:56.292829 14093 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:30:56.293183 14093 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:30:56.294497 14093 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:30:56.296088 14093 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:30:56.296172 14093 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:30:56.296329 14093 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:30:56.296391 14093 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:30:56.331990 14093 rpc_server.cc:307] RPC server started. Bound to: 127.8.18.193:36441
I20260810 11:30:56.332096 14229 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.18.193:36441 every 8 connection(s)
I20260810 11:30:56.333528 14093 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786361401922362-8267-0/minicluster-data/ts-0/data/info.pb
I20260810 11:30:56.342131 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 14093
I20260810 11:30:56.342402 8267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786361401922362-8267-0/minicluster-data/ts-0/wal/instance
I20260810 11:30:56.343262 8267 external_mini_cluster.cc:1428] Setting key 4af721358080d55e276e853e68e7df9a
Aug 10 11:30:56 dist-test-slave-srgv krb5kdc[13976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361456, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.193@KRBTEST.COM for kudu/127.8.18.254@KRBTEST.COM
I20260810 11:30:56.359387 14232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:56.336724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.193:52399 (local address 127.8.18.254:35451)
0810 11:30:56.337281 (+ 557us) server_negotiation.cc:206] Beginning negotiation
0810 11:30:56.337292 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 11:30:56.338519 (+ 1227us) server_negotiation.cc:407] Connection header received
0810 11:30:56.340576 (+ 2057us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:30:56.340582 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:30:56.340680 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:30:56.340827 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:30:56.343252 (+ 2425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:56.344255 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:56.346560 (+ 2305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:56.346988 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:56.347103 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:56.352198 (+ 5095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:30:56.352203 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:30:56.352207 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:30:56.352248 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:30:56.355166 (+ 2918us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:56.356323 (+ 1157us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:56.356328 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:56.356331 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:56.356471 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:56.357045 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:56.357049 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:56.357051 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:56.357380 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:30:56.357600 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0810 11:30:56.358722 (+ 1122us) server_negotiation.cc:299] Negotiation successful
0810 11:30:56.359028 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":430,"thread_start_us":278,"threads_started":1}
I20260810 11:30:56.360949 14233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:56.337391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:35451 (local address 127.8.18.193:52399)
0810 11:30:56.338309 (+ 918us) negotiation.cc:107] Waiting for socket to connect
0810 11:30:56.338361 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0810 11:30:56.340230 (+ 1869us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:30:56.341111 (+ 881us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:30:56.341120 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:30:56.341999 (+ 879us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:30:56.342986 (+ 987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:30:56.343019 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:30:56.344454 (+ 1435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:30:56.344460 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:30:56.346382 (+ 1922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:30:56.346400 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:30:56.347269 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:30:56.347280 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:30:56.347677 (+ 397us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:56.348529 (+ 852us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:30:56.348572 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:30:56.351888 (+ 3316us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:30:56.355364 (+ 3476us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:30:56.355380 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:30:56.355403 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:30:56.356134 (+ 731us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:30:56.356602 (+ 468us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:30:56.356611 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:30:56.356617 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:30:56.356912 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:30:56.357587 (+ 675us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:30:56.357599 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:30:56.358158 (+ 559us) client_negotiation.cc:769] Sending connection context
0810 11:30:56.358537 (+ 379us) client_negotiation.cc:240] Negotiation successful
0810 11:30:56.359249 (+ 712us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":586,"thread_start_us":341,"threads_started":1}
I20260810 11:30:56.362918 14230 heartbeater.cc:347] Connected to a master server at 127.8.18.254:35451
I20260810 11:30:56.363319 14230 heartbeater.cc:464] Registering TS with master...
I20260810 11:30:56.364562 14230 heartbeater.cc:511] Master 127.8.18.254:35451 requested a full tablet report, sending...
I20260810 11:30:56.367563 14024 ts_manager.cc:194] Registered new tserver with Master: ac22ecce71aa4b2980b3ff051fd0b344 (127.8.18.193:36441)
I20260810 11:30:56.369263 14024 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.18.193@KRBTEST.COM'} at 127.8.18.193:52399
WARNING: no policy specified for kudu/127.8.18.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.18.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.18.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.194.keytab.
Entry for principal kudu/127.8.18.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.194.keytab.
I20260810 11:30:56.405591 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786361401922362-8267-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786361401922362-8267-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786361401922362-8267-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786361401922362-8267-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.18.194:0
--local_ip_for_outbound_sockets=127.8.18.194
--webserver_interface=127.8.18.194
--webserver_port=0
--tserver_master_addrs=127.8.18.254:35451
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.194.keytab
--principal=kudu/127.8.18.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.18.212:39391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786361401922362-8267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786361401922362-8267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786361401922362-8267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:30:56.779347 14236 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:30:56.779719 14236 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:30:56.779901 14236 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:30:56.787675 14236 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:30:56.787843 14236 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:30:56.788040 14236 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.18.194
I20260810 11:30:56.798437 14236 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.18.212:39391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786361401922362-8267-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786361401922362-8267-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.194.keytab
--principal=kudu/127.8.18.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.18.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786361401922362-8267-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.18.194
--webserver_port=0
--tserver_master_addrs=127.8.18.254:35451
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.18.194
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786361401922362-8267-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:30:56.800014 14236 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:30:56.801517 14236 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:30:56.816327 14244 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:30:56.816381 14241 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:30:56.816601 14242 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:30:56.817597 14236 server_base.cc:1034] running on GCE node
I20260810 11:30:56.818336 14236 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:30:56.819739 14236 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:30:56.821094 14236 hybrid_clock.cc:648] HybridClock initialized: now 1786361456821009 us; error 93 us; skew 500 ppm
I20260810 11:30:56.821668 14236 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:30:56 dist-test-slave-srgv krb5kdc[13976](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361456, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:30:56.825291 14236 init.cc:377] Logged in from keytab as kudu/127.8.18.194@KRBTEST.COM (short username kudu)
I20260810 11:30:56.827183 14236 webserver.cc:460] Webserver started at http://127.8.18.194:39797/ using document root <none> and password file <none>
I20260810 11:30:56.827930 14236 fs_manager.cc:359] Metadata directory not provided
I20260810 11:30:56.828047 14236 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:30:56.828334 14236 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:30:56.831619 14236 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786361401922362-8267-0/minicluster-data/ts-1/data/instance:
uuid: "e20ccf1c45ff4d958280e7866f162ad1"
format_stamp: "Formatted at 2026-08-10 11:30:56 on dist-test-slave-srgv"
server_key: "4c89941f8f4e4d919739e7a786181f10"
server_key_iv: "d64c2d80cb8ec91793675772ec038c67"
server_key_version: "encryptionkey@0"
I20260810 11:30:56.832484 14236 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786361401922362-8267-0/minicluster-data/ts-1/wal/instance:
uuid: "e20ccf1c45ff4d958280e7866f162ad1"
format_stamp: "Formatted at 2026-08-10 11:30:56 on dist-test-slave-srgv"
server_key: "4c89941f8f4e4d919739e7a786181f10"
server_key_iv: "d64c2d80cb8ec91793675772ec038c67"
server_key_version: "encryptionkey@0"
I20260810 11:30:56.838584 14236 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260810 11:30:56.842455 14251 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:30:56.844310 14236 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 11:30:56.844558 14236 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786361401922362-8267-0/minicluster-data/ts-1/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786361401922362-8267-0/minicluster-data/ts-1/wal
uuid: "e20ccf1c45ff4d958280e7866f162ad1"
format_stamp: "Formatted at 2026-08-10 11:30:56 on dist-test-slave-srgv"
server_key: "4c89941f8f4e4d919739e7a786181f10"
server_key_iv: "d64c2d80cb8ec91793675772ec038c67"
server_key_version: "encryptionkey@0"
I20260810 11:30:56.844838 14236 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786361401922362-8267-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786361401922362-8267-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786361401922362-8267-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:30:56.866547 14236 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:30:56.871690 14236 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:30:56.872041 14236 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:30:56.873162 14236 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:30:56.874832 14236 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:30:56.874915 14236 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:30:56.875097 14236 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:30:56.875165 14236 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:30:56.911835 14236 rpc_server.cc:307] RPC server started. Bound to: 127.8.18.194:45557
I20260810 11:30:56.911981 14369 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.18.194:45557 every 8 connection(s)
I20260810 11:30:56.913316 14236 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786361401922362-8267-0/minicluster-data/ts-1/data/info.pb
I20260810 11:30:56.922426 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 14236
I20260810 11:30:56.922668 8267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786361401922362-8267-0/minicluster-data/ts-1/wal/instance
I20260810 11:30:56.923156 8267 external_mini_cluster.cc:1428] Setting key 66a3be35a56467bbbd13cd8dac32353a
Aug 10 11:30:56 dist-test-slave-srgv krb5kdc[13976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361456, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.194@KRBTEST.COM for kudu/127.8.18.254@KRBTEST.COM
I20260810 11:30:56.935700 14372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:56.916328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.194:38481 (local address 127.8.18.254:35451)
0810 11:30:56.916856 (+ 528us) server_negotiation.cc:206] Beginning negotiation
0810 11:30:56.916863 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:30:56.917957 (+ 1094us) server_negotiation.cc:407] Connection header received
0810 11:30:56.919837 (+ 1880us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:30:56.919842 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:30:56.919916 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:30:56.920026 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:30:56.921898 (+ 1872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:56.922747 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:56.924707 (+ 1960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:56.925260 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:56.925414 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:56.929247 (+ 3833us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:30:56.929253 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:30:56.929256 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:30:56.929293 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:30:56.932121 (+ 2828us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:56.933035 (+ 914us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:56.933040 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:56.933043 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:56.933188 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:56.933765 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:56.933769 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:56.933771 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:56.934073 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:30:56.934249 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0810 11:30:56.935032 (+ 783us) server_negotiation.cc:299] Negotiation successful
0810 11:30:56.935291 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":376,"thread_start_us":247,"threads_started":1}
I20260810 11:30:56.936817 14373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:56.916852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:35451 (local address 127.8.18.194:38481)
0810 11:30:56.917773 (+ 921us) negotiation.cc:107] Waiting for socket to connect
0810 11:30:56.917815 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 11:30:56.919528 (+ 1713us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:30:56.920348 (+ 820us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:30:56.920357 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:30:56.920938 (+ 581us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:30:56.921701 (+ 763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:30:56.921723 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:30:56.922938 (+ 1215us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:30:56.922943 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:30:56.924538 (+ 1595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:30:56.924553 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:30:56.925553 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:30:56.925562 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:30:56.925812 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:56.926559 (+ 747us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:30:56.926588 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:30:56.928935 (+ 2347us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:30:56.932328 (+ 3393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:30:56.932340 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:30:56.932367 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:30:56.932861 (+ 494us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:30:56.933319 (+ 458us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:30:56.933325 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:30:56.933329 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:30:56.933617 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:30:56.934218 (+ 601us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:30:56.934235 (+ 17us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:30:56.934629 (+ 394us) client_negotiation.cc:769] Sending connection context
0810 11:30:56.934930 (+ 301us) client_negotiation.cc:240] Negotiation successful
0810 11:30:56.935559 (+ 629us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":651,"thread_start_us":344,"threads_started":1}
I20260810 11:30:56.938596 14370 heartbeater.cc:347] Connected to a master server at 127.8.18.254:35451
I20260810 11:30:56.939023 14370 heartbeater.cc:464] Registering TS with master...
I20260810 11:30:56.940145 14370 heartbeater.cc:511] Master 127.8.18.254:35451 requested a full tablet report, sending...
I20260810 11:30:56.942126 14024 ts_manager.cc:194] Registered new tserver with Master: e20ccf1c45ff4d958280e7866f162ad1 (127.8.18.194:45557)
I20260810 11:30:56.943073 14024 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.18.194@KRBTEST.COM'} at 127.8.18.194:38481
WARNING: no policy specified for kudu/127.8.18.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.18.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.18.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.195.keytab.
Entry for principal kudu/127.8.18.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.195.keytab.
I20260810 11:30:56.982461 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786361401922362-8267-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786361401922362-8267-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786361401922362-8267-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786361401922362-8267-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.18.195:0
--local_ip_for_outbound_sockets=127.8.18.195
--webserver_interface=127.8.18.195
--webserver_port=0
--tserver_master_addrs=127.8.18.254:35451
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.195.keytab
--principal=kudu/127.8.18.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.18.212:39391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786361401922362-8267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786361401922362-8267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786361401922362-8267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:30:57.346221 14376 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:30:57.346573 14376 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:30:57.346679 14376 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:30:57.354944 14376 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:30:57.355085 14376 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:30:57.355243 14376 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.18.195
I20260810 11:30:57.366318 14376 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.18.212:39391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786361401922362-8267-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786361401922362-8267-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.195.keytab
--principal=kudu/127.8.18.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.18.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786361401922362-8267-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.18.195
--webserver_port=0
--tserver_master_addrs=127.8.18.254:35451
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.18.195
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786361401922362-8267-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:30:57.367740 14376 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:30:57.369149 14376 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 11:30:57.373188 14230 heartbeater.cc:503] Master 127.8.18.254:35451 was elected leader, sending a full tablet report...
W20260810 11:30:57.385345 14381 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:30:57.385429 14384 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:30:57.385286 14382 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:30:57.386505 14376 server_base.cc:1034] running on GCE node
I20260810 11:30:57.387245 14376 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:30:57.388574 14376 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:30:57.389833 14376 hybrid_clock.cc:648] HybridClock initialized: now 1786361457389787 us; error 46 us; skew 500 ppm
I20260810 11:30:57.390224 14376 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:30:57 dist-test-slave-srgv krb5kdc[13976](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361457, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:30:57.394394 14376 init.cc:377] Logged in from keytab as kudu/127.8.18.195@KRBTEST.COM (short username kudu)
I20260810 11:30:57.396324 14376 webserver.cc:460] Webserver started at http://127.8.18.195:33289/ using document root <none> and password file <none>
I20260810 11:30:57.397086 14376 fs_manager.cc:359] Metadata directory not provided
I20260810 11:30:57.397208 14376 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:30:57.397583 14376 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:30:57.401115 14376 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786361401922362-8267-0/minicluster-data/ts-2/data/instance:
uuid: "9fbc6c58c6a14f0797183ba8c8f50080"
format_stamp: "Formatted at 2026-08-10 11:30:57 on dist-test-slave-srgv"
server_key: "56d020eca82c855bcba3445e7fc60295"
server_key_iv: "e8a668fc5f219cbc3db74a576b884b3d"
server_key_version: "encryptionkey@0"
I20260810 11:30:57.402097 14376 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786361401922362-8267-0/minicluster-data/ts-2/wal/instance:
uuid: "9fbc6c58c6a14f0797183ba8c8f50080"
format_stamp: "Formatted at 2026-08-10 11:30:57 on dist-test-slave-srgv"
server_key: "56d020eca82c855bcba3445e7fc60295"
server_key_iv: "e8a668fc5f219cbc3db74a576b884b3d"
server_key_version: "encryptionkey@0"
I20260810 11:30:57.408247 14376 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260810 11:30:57.412616 14392 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:30:57.414490 14376 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260810 11:30:57.414703 14376 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786361401922362-8267-0/minicluster-data/ts-2/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786361401922362-8267-0/minicluster-data/ts-2/wal
uuid: "9fbc6c58c6a14f0797183ba8c8f50080"
format_stamp: "Formatted at 2026-08-10 11:30:57 on dist-test-slave-srgv"
server_key: "56d020eca82c855bcba3445e7fc60295"
server_key_iv: "e8a668fc5f219cbc3db74a576b884b3d"
server_key_version: "encryptionkey@0"
I20260810 11:30:57.414932 14376 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786361401922362-8267-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786361401922362-8267-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786361401922362-8267-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:30:57.446022 14376 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:30:57.451223 14376 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:30:57.451581 14376 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:30:57.452755 14376 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:30:57.454306 14376 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:30:57.454433 14376 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:30:57.454608 14376 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:30:57.454715 14376 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:30:57.490885 14376 rpc_server.cc:307] RPC server started. Bound to: 127.8.18.195:38417
I20260810 11:30:57.490943 14514 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.18.195:38417 every 8 connection(s)
I20260810 11:30:57.492364 14376 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786361401922362-8267-0/minicluster-data/ts-2/data/info.pb
I20260810 11:30:57.500531 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 14376
I20260810 11:30:57.500774 8267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786361401922362-8267-0/minicluster-data/ts-2/wal/instance
I20260810 11:30:57.501240 8267 external_mini_cluster.cc:1428] Setting key 7cfa0ac68206af71e1896e7455ec28bf
Aug 10 11:30:57 dist-test-slave-srgv krb5kdc[13976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361457, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.195@KRBTEST.COM for kudu/127.8.18.254@KRBTEST.COM
I20260810 11:30:57.518738 14517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:57.495927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.195:58687 (local address 127.8.18.254:35451)
0810 11:30:57.496517 (+ 590us) server_negotiation.cc:206] Beginning negotiation
0810 11:30:57.496522 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:30:57.497795 (+ 1273us) server_negotiation.cc:407] Connection header received
0810 11:30:57.499946 (+ 2151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:30:57.499950 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:30:57.500113 (+ 163us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:30:57.500262 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:30:57.502474 (+ 2212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:57.503660 (+ 1186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:57.505871 (+ 2211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:57.506322 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:57.506437 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:57.510848 (+ 4411us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:30:57.510854 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:30:57.510858 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:30:57.510899 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:30:57.514186 (+ 3287us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:57.515441 (+ 1255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:57.515446 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:57.515449 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:57.515619 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:57.516381 (+ 762us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:57.516389 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:57.516392 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:57.516835 (+ 443us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:30:57.517030 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0810 11:30:57.518133 (+ 1103us) server_negotiation.cc:299] Negotiation successful
0810 11:30:57.518418 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":465,"thread_start_us":323,"threads_started":1}
I20260810 11:30:57.520085 14518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:57.496674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:35451 (local address 127.8.18.195:58687)
0810 11:30:57.497586 (+ 912us) negotiation.cc:107] Waiting for socket to connect
0810 11:30:57.497654 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0810 11:30:57.499606 (+ 1952us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:30:57.500635 (+ 1029us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:30:57.500645 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:30:57.501469 (+ 824us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:30:57.502230 (+ 761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:30:57.502260 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:30:57.503914 (+ 1654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:30:57.503918 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:30:57.505705 (+ 1787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:30:57.505721 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:30:57.506595 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:30:57.506601 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:30:57.506858 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:57.507604 (+ 746us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:30:57.507644 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:30:57.510499 (+ 2855us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:30:57.514495 (+ 3996us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:30:57.514506 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:30:57.514524 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:30:57.515159 (+ 635us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:30:57.515816 (+ 657us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:30:57.515820 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:30:57.515823 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:30:57.516183 (+ 360us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:30:57.517002 (+ 819us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:30:57.517011 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:30:57.517605 (+ 594us) client_negotiation.cc:769] Sending connection context
0810 11:30:57.517945 (+ 340us) client_negotiation.cc:240] Negotiation successful
0810 11:30:57.518602 (+ 657us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":576,"thread_start_us":353,"threads_started":1}
I20260810 11:30:57.522055 14515 heartbeater.cc:347] Connected to a master server at 127.8.18.254:35451
I20260810 11:30:57.522470 14515 heartbeater.cc:464] Registering TS with master...
I20260810 11:30:57.523394 14515 heartbeater.cc:511] Master 127.8.18.254:35451 requested a full tablet report, sending...
I20260810 11:30:57.525573 14024 ts_manager.cc:194] Registered new tserver with Master: 9fbc6c58c6a14f0797183ba8c8f50080 (127.8.18.195:38417)
I20260810 11:30:57.526463 14024 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.18.195@KRBTEST.COM'} at 127.8.18.195:58687
I20260810 11:30:57.531454 8267 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786361401922362-8267-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786361401922362-8267-0/krb5kdc/test-admin.keytab.
Aug 10 11:30:57 dist-test-slave-srgv krb5kdc[13976](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361457, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:30:57.564162 8267 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260810 11:30:57.570371 14524 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:30:57.567407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33318 (local address 127.8.18.193:36441)
0810 11:30:57.568129 (+ 722us) server_negotiation.cc:206] Beginning negotiation
0810 11:30:57.568137 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:30:57.568164 (+ 27us) server_negotiation.cc:407] Connection header received
0810 11:30:57.568223 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:30:57.568230 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:30:57.568469 (+ 239us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:30:57.568653 (+ 184us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:30:57.569924 (+ 1271us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33318: BlockingRecv error: recv got EOF from 127.0.0.1:33318 (error 108)
Metrics: {"server-negotiator.queue_time_us":509,"thread_start_us":343,"threads_started":1}
W20260810 11:30:57.570631 8267 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786361401922362-8267-0/krb5kdc/krb5cc' of type FILE
Aug 10 11:30:57 dist-test-slave-srgv krb5kdc[13976](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361457, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:30:57.572607 8267 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 10 11:30:57 dist-test-slave-srgv krb5kdc[13976](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361457, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:30:57.574259 8267 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 10 11:30:57 dist-test-slave-srgv krb5kdc[13976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361457, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.18.193@KRBTEST.COM
I20260810 11:30:57.592984 14524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:57.576378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33320 (local address 127.8.18.193:36441)
0810 11:30:57.576627 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0810 11:30:57.576634 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:30:57.576903 (+ 269us) server_negotiation.cc:407] Connection header received
0810 11:30:57.577067 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:30:57.577071 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:30:57.577136 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:30:57.577234 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:30:57.578622 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:57.579654 (+ 1032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:57.580975 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:57.581684 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:57.581809 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:57.586205 (+ 4396us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:30:57.586222 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:30:57.586238 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:30:57.586296 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:30:57.589902 (+ 3606us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:57.590785 (+ 883us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:57.590794 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:57.590796 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:57.590945 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:57.591360 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:57.591364 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:57.591366 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:57.591784 (+ 418us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:30:57.592025 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0810 11:30:57.592341 (+ 316us) server_negotiation.cc:299] Negotiation successful
0810 11:30:57.592634 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
W20260810 11:30:57.600021 14524 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:30:57.598038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33332 (local address 127.8.18.193:36441)
0810 11:30:57.598257 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0810 11:30:57.598264 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:30:57.598458 (+ 194us) server_negotiation.cc:407] Connection header received
0810 11:30:57.598610 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:30:57.598613 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:30:57.598678 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:30:57.598802 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:30:57.599723 (+ 921us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33332: BlockingRecv error: recv got EOF from 127.0.0.1:33332 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
W20260810 11:30:57.600181 8267 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786361401922362-8267-0/krb5kdc/krb5cc' of type FILE
Aug 10 11:30:57 dist-test-slave-srgv krb5kdc[13976](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361457, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:30:57.602202 8267 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 10 11:30:57 dist-test-slave-srgv krb5kdc[13976](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361457, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:30:57.603756 8267 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 10 11:30:57 dist-test-slave-srgv krb5kdc[13976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361457, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.18.193@KRBTEST.COM
I20260810 11:30:57.621078 14524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:57.605375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33346 (local address 127.8.18.193:36441)
0810 11:30:57.605660 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0810 11:30:57.605667 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:30:57.605968 (+ 301us) server_negotiation.cc:407] Connection header received
0810 11:30:57.606118 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:30:57.606122 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:30:57.606188 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:30:57.606284 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:30:57.607453 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:57.608272 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:57.609722 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:57.610172 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:57.610294 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:57.614804 (+ 4510us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:30:57.614810 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:30:57.614814 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:30:57.614868 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:30:57.617900 (+ 3032us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:57.618801 (+ 901us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:57.618806 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:57.618808 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:57.618952 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:57.619468 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:57.619472 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:57.619474 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:57.619810 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:30:57.620104 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0810 11:30:57.620630 (+ 526us) server_negotiation.cc:299] Negotiation successful
0810 11:30:57.620844 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105}
W20260810 11:30:57.626659 14524 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:30:57.624155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33352 (local address 127.8.18.193:36441)
0810 11:30:57.624435 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0810 11:30:57.624442 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:30:57.624968 (+ 526us) server_negotiation.cc:407] Connection header received
0810 11:30:57.625215 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:30:57.625219 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:30:57.625297 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:30:57.625408 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:30:57.626351 (+ 943us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33352: BlockingRecv error: recv got EOF from 127.0.0.1:33352 (error 108)
Metrics: {"server-negotiator.queue_time_us":121}
W20260810 11:30:57.626950 8267 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786361401922362-8267-0/krb5kdc/krb5cc' of type FILE
Aug 10 11:30:57 dist-test-slave-srgv krb5kdc[13976](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361457, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:30:57.628782 8267 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 10 11:30:57 dist-test-slave-srgv krb5kdc[13976](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361457, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:30:57.630419 8267 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 10 11:30:57 dist-test-slave-srgv krb5kdc[13976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361457, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.18.193@KRBTEST.COM
I20260810 11:30:57.648032 14524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:57.632020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33364 (local address 127.8.18.193:36441)
0810 11:30:57.632272 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0810 11:30:57.632281 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 11:30:57.632575 (+ 294us) server_negotiation.cc:407] Connection header received
0810 11:30:57.632798 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:30:57.632804 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:30:57.632883 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:30:57.632997 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:30:57.634272 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:57.635356 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:57.636780 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:57.637483 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:57.637625 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:57.641802 (+ 4177us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:30:57.641809 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:30:57.641815 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:30:57.641865 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:30:57.644975 (+ 3110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:57.646015 (+ 1040us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:57.646020 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:57.646023 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:57.646227 (+ 204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:57.646615 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:57.646619 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:57.646621 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:57.646996 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:30:57.647300 (+ 304us) server_negotiation.cc:1035] Waiting for connection context
0810 11:30:57.647551 (+ 251us) server_negotiation.cc:299] Negotiation successful
0810 11:30:57.647753 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260810 11:30:57.649940 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 14093
I20260810 11:30:57.664354 14524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:57.650796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33378 (local address 127.8.18.193:36441)
0810 11:30:57.651031 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0810 11:30:57.651039 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:30:57.651359 (+ 320us) server_negotiation.cc:407] Connection header received
0810 11:30:57.651505 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:30:57.651508 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:30:57.651571 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:30:57.651655 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:30:57.652734 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:57.653531 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:57.655106 (+ 1575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:57.655564 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:57.655670 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:57.658244 (+ 2574us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:30:57.658250 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:30:57.658253 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:30:57.658299 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:30:57.661231 (+ 2932us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:57.662203 (+ 972us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:57.662208 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:57.662211 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:57.662383 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:57.662810 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:57.662813 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:57.662815 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:57.663121 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:30:57.663346 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0810 11:30:57.663656 (+ 310us) server_negotiation.cc:299] Negotiation successful
0810 11:30:57.663876 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260810 11:30:57.665021 14225 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:30:57.753623 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 14093
I20260810 11:30:57.772045 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 14236
Aug 10 11:30:57 dist-test-slave-srgv krb5kdc[13976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361457, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.18.194@KRBTEST.COM
I20260810 11:30:57.789811 14537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:57.773048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48842 (local address 127.8.18.194:45557)
0810 11:30:57.773702 (+ 654us) server_negotiation.cc:206] Beginning negotiation
0810 11:30:57.773709 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:30:57.773738 (+ 29us) server_negotiation.cc:407] Connection header received
0810 11:30:57.773802 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:30:57.773808 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:30:57.774025 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:30:57.774216 (+ 191us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:30:57.775370 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:57.776448 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:57.777958 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:57.778590 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:57.778765 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:57.783157 (+ 4392us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:30:57.783175 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:30:57.783191 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:30:57.783250 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:30:57.786432 (+ 3182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:57.787294 (+ 862us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:57.787306 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:57.787314 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:57.787494 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:57.787925 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:57.787930 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:57.787934 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:57.788329 (+ 395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:30:57.788668 (+ 339us) server_negotiation.cc:1035] Waiting for connection context
0810 11:30:57.789055 (+ 387us) server_negotiation.cc:299] Negotiation successful
0810 11:30:57.789358 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":504,"thread_start_us":291,"threads_started":1}
I20260810 11:30:57.791312 14364 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:30:57.876675 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 14236
I20260810 11:30:57.895074 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 14376
Aug 10 11:30:57 dist-test-slave-srgv krb5kdc[13976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361457, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.18.195@KRBTEST.COM
I20260810 11:30:57.915427 14539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:30:57.896359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59514 (local address 127.8.18.195:38417)
0810 11:30:57.897123 (+ 764us) server_negotiation.cc:206] Beginning negotiation
0810 11:30:57.897131 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:30:57.897212 (+ 81us) server_negotiation.cc:407] Connection header received
0810 11:30:57.897290 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:30:57.897298 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:30:57.897655 (+ 357us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:30:57.897888 (+ 233us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:30:57.899299 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:57.900502 (+ 1203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:57.902116 (+ 1614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:30:57.902796 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:30:57.902972 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:30:57.907970 (+ 4998us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:30:57.907988 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:30:57.908006 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:30:57.908053 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:30:57.911549 (+ 3496us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:57.912572 (+ 1023us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:57.912593 (+ 21us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:57.912595 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:57.912773 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:30:57.913568 (+ 795us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:30:57.913574 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:30:57.913576 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:30:57.914084 (+ 508us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:30:57.914379 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0810 11:30:57.914811 (+ 432us) server_negotiation.cc:299] Negotiation successful
0810 11:30:57.915077 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":578,"thread_start_us":399,"threads_started":1}
I20260810 11:30:57.916960 14509 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:30:58.007236 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 14376
I20260810 11:30:58.024783 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 13990
I20260810 11:30:58.026527 14058 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:30:58.133770 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 13990
2026-08-10T11:30:58Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (4393 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 11:30:58 dist-test-slave-srgv krb5kdc[14545](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 11:30:58 dist-test-slave-srgv krb5kdc[14545](info): set up 2 sockets
Aug 10 11:30:58 dist-test-slave-srgv krb5kdc[14545](info): commencing operation
krb5kdc: starting...
W20260810 11:30:59.274266 8267 mini_kdc.cc:121] Time spent starting KDC: real 1.094s user 0.004s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/krb5kdc/test-user.keytab.
Aug 10 11:30:59 dist-test-slave-srgv krb5kdc[14545](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361459, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T11:30:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T11:30:59Z Disabled control of system clock
WARNING: no policy specified for oryx/127.8.18.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.18.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.18.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/krb5kdc/oryx_127.8.18.254.keytab.
Entry for principal oryx/127.8.18.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/krb5kdc/oryx_127.8.18.254.keytab.
I20260810 11:30:59.545971 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.18.254:36523
--webserver_interface=127.8.18.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/krb5kdc/oryx_127.8.18.254.keytab
--principal=oryx/127.8.18.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.18.212:34803
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.18.254:36523
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:30:59.912504 14559 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:30:59.912870 14559 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:30:59.913039 14559 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:30:59.920858 14559 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 11:30:59.921087 14559 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:30:59.921195 14559 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:30:59.921303 14559 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 11:30:59.921389 14559 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 11:30:59.933106 14559 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.18.212:34803
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.18.254:36523
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/krb5kdc/oryx_127.8.18.254.keytab
--principal=oryx/127.8.18.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.18.254:36523
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.18.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:30:59.934754 14559 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:30:59.936270 14559 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:30:59.951884 14565 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:30:59.951943 14564 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:30:59.951843 14567 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:30:59.952970 14559 server_base.cc:1034] running on GCE node
I20260810 11:30:59.954216 14559 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:30:59.956331 14559 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:30:59.957598 14559 hybrid_clock.cc:648] HybridClock initialized: now 1786361459957555 us; error 71 us; skew 500 ppm
I20260810 11:30:59.958048 14559 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:30:59 dist-test-slave-srgv krb5kdc[14545](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361459, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.18.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:30:59.961946 14559 init.cc:377] Logged in from keytab as oryx/127.8.18.254@KRBTEST.COM (short username oryx)
I20260810 11:30:59.963862 14559 webserver.cc:460] Webserver started at http://127.8.18.254:33103/ using document root <none> and password file <none>
I20260810 11:30:59.964713 14559 fs_manager.cc:359] Metadata directory not provided
I20260810 11:30:59.964905 14559 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:30:59.965283 14559 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:30:59.968834 14559 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/master-0/data/instance:
uuid: "cd8ee5823e444d28a6cb100898b494c5"
format_stamp: "Formatted at 2026-08-10 11:30:59 on dist-test-slave-srgv"
server_key: "e3b5da0519c62a5102836e8db4461072"
server_key_iv: "ff7864307058569d86cc748502055bbb"
server_key_version: "encryptionkey@0"
I20260810 11:30:59.969918 14559 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/master-0/wal/instance:
uuid: "cd8ee5823e444d28a6cb100898b494c5"
format_stamp: "Formatted at 2026-08-10 11:30:59 on dist-test-slave-srgv"
server_key: "e3b5da0519c62a5102836e8db4461072"
server_key_iv: "ff7864307058569d86cc748502055bbb"
server_key_version: "encryptionkey@0"
I20260810 11:30:59.976336 14559 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.001s sys 0.005s
I20260810 11:30:59.980873 14574 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:30:59.982887 14559 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.003s
I20260810 11:30:59.983175 14559 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/master-0/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/master-0/wal
uuid: "cd8ee5823e444d28a6cb100898b494c5"
format_stamp: "Formatted at 2026-08-10 11:30:59 on dist-test-slave-srgv"
server_key: "e3b5da0519c62a5102836e8db4461072"
server_key_iv: "ff7864307058569d86cc748502055bbb"
server_key_version: "encryptionkey@0"
I20260810 11:30:59.983438 14559 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:31:00.014617 14559 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:31:00.020053 14559 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:31:00.020440 14559 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:31:00.042344 14559 rpc_server.cc:307] RPC server started. Bound to: 127.8.18.254:36523
I20260810 11:31:00.042457 14626 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.18.254:36523 every 8 connection(s)
I20260810 11:31:00.043978 14559 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
I20260810 11:31:00.048419 14627 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:31:00.054399 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 14559
I20260810 11:31:00.054652 8267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/master-0/wal/instance
I20260810 11:31:00.055511 8267 external_mini_cluster.cc:1428] Setting key c99ff02f33ec007b28a944a79e6c3a58
I20260810 11:31:00.061717 14627 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd8ee5823e444d28a6cb100898b494c5: Bootstrap starting.
I20260810 11:31:00.066049 14627 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cd8ee5823e444d28a6cb100898b494c5: Neither blocks nor log segments found. Creating new log.
I20260810 11:31:00.067592 14627 log.cc:824] T 00000000000000000000000000000000 P cd8ee5823e444d28a6cb100898b494c5: Log is configured to *not* fsync() on all Append() calls
I20260810 11:31:00.072131 14627 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd8ee5823e444d28a6cb100898b494c5: No bootstrap required, opened a new log
Aug 10 11:31:00 dist-test-slave-srgv krb5kdc[14545](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361459, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.8.18.254@KRBTEST.COM
I20260810 11:31:00.081102 14627 raft_consensus.cc:348] T 00000000000000000000000000000000 P cd8ee5823e444d28a6cb100898b494c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd8ee5823e444d28a6cb100898b494c5" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 36523 } }
I20260810 11:31:00.081624 14627 raft_consensus.cc:374] T 00000000000000000000000000000000 P cd8ee5823e444d28a6cb100898b494c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:31:00.081785 14627 raft_consensus.cc:729] T 00000000000000000000000000000000 P cd8ee5823e444d28a6cb100898b494c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd8ee5823e444d28a6cb100898b494c5, State: Initialized, Role: FOLLOWER
I20260810 11:31:00.082610 14627 consensus_queue.cc:260] T 00000000000000000000000000000000 P cd8ee5823e444d28a6cb100898b494c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd8ee5823e444d28a6cb100898b494c5" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 36523 } }
I20260810 11:31:00.082912 14627 raft_consensus.cc:388] T 00000000000000000000000000000000 P cd8ee5823e444d28a6cb100898b494c5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 11:31:00.083109 14627 raft_consensus.cc:482] T 00000000000000000000000000000000 P cd8ee5823e444d28a6cb100898b494c5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 11:31:00.083325 14627 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cd8ee5823e444d28a6cb100898b494c5 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:31:00.084316 14630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:00.058154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40410 (local address 127.8.18.254:36523)
0810 11:31:00.059220 (+ 1066us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:00.059235 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:00.059290 (+ 55us) server_negotiation.cc:407] Connection header received
0810 11:31:00.061260 (+ 1970us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:00.061292 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:00.061864 (+ 572us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:00.062280 (+ 416us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:31:00.064513 (+ 2233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:00.066021 (+ 1508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:00.068023 (+ 2002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:00.069365 (+ 1342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:00.069637 (+ 272us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:00.075074 (+ 5437us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:31:00.075084 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:31:00.075107 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:31:00.075158 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:31:00.078991 (+ 3833us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:31:00.079917 (+ 926us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:31:00.079930 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:31:00.079942 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:31:00.080166 (+ 224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:31:00.080632 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:31:00.080640 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:31:00.080647 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:31:00.081306 (+ 659us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:31:00.081760 (+ 454us) server_negotiation.cc:1035] Waiting for connection context
0810 11:31:00.082245 (+ 485us) server_negotiation.cc:299] Negotiation successful
0810 11:31:00.082805 (+ 560us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":806,"thread_start_us":551,"threads_started":1}
I20260810 11:31:00.086752 14627 raft_consensus.cc:504] T 00000000000000000000000000000000 P cd8ee5823e444d28a6cb100898b494c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd8ee5823e444d28a6cb100898b494c5" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 36523 } }
I20260810 11:31:00.087459 14627 leader_election.cc:302] T 00000000000000000000000000000000 P cd8ee5823e444d28a6cb100898b494c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cd8ee5823e444d28a6cb100898b494c5; no voters:
I20260810 11:31:00.088290 14627 leader_election.cc:288] T 00000000000000000000000000000000 P cd8ee5823e444d28a6cb100898b494c5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 11:31:00.088442 14632 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cd8ee5823e444d28a6cb100898b494c5 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:31:00.089190 14632 raft_consensus.cc:686] T 00000000000000000000000000000000 P cd8ee5823e444d28a6cb100898b494c5 [term 1 LEADER]: Becoming Leader. State: Replica: cd8ee5823e444d28a6cb100898b494c5, State: Running, Role: LEADER
I20260810 11:31:00.089823 14632 consensus_queue.cc:237] T 00000000000000000000000000000000 P cd8ee5823e444d28a6cb100898b494c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd8ee5823e444d28a6cb100898b494c5" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 36523 } }
I20260810 11:31:00.090634 14627 sys_catalog.cc:564] T 00000000000000000000000000000000 P cd8ee5823e444d28a6cb100898b494c5 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 11:31:00.094439 14634 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd8ee5823e444d28a6cb100898b494c5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cd8ee5823e444d28a6cb100898b494c5. Latest consensus state: current_term: 1 leader_uuid: "cd8ee5823e444d28a6cb100898b494c5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd8ee5823e444d28a6cb100898b494c5" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 36523 } } }
I20260810 11:31:00.094369 14633 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd8ee5823e444d28a6cb100898b494c5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cd8ee5823e444d28a6cb100898b494c5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd8ee5823e444d28a6cb100898b494c5" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 36523 } } }
I20260810 11:31:00.094988 14633 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd8ee5823e444d28a6cb100898b494c5 [sys.catalog]: This master's current role is: LEADER
I20260810 11:31:00.095867 14634 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd8ee5823e444d28a6cb100898b494c5 [sys.catalog]: This master's current role is: LEADER
I20260810 11:31:00.097764 14644 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 11:31:00.103354 14644 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 11:31:00.115538 14644 catalog_manager.cc:1347] Generated new cluster ID: 63349ed0186a4282894be80560e2d468
I20260810 11:31:00.115700 14644 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 11:31:00.135967 14644 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 11:31:00.138154 14644 catalog_manager.cc:1504] Loading token signing keys...
I20260810 11:31:00.157505 14644 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cd8ee5823e444d28a6cb100898b494c5: Generated new TSK 0
I20260810 11:31:00.158727 14644 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.8.18.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.18.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.18.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/krb5kdc/oryx_127.8.18.193.keytab.
Entry for principal oryx/127.8.18.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/krb5kdc/oryx_127.8.18.193.keytab.
I20260810 11:31:00.226878 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.18.193:0
--local_ip_for_outbound_sockets=127.8.18.193
--webserver_interface=127.8.18.193
--webserver_port=0
--tserver_master_addrs=127.8.18.254:36523
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/krb5kdc/oryx_127.8.18.193.keytab
--principal=oryx/127.8.18.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.18.212:34803
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:31:00.621037 14655 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:31:00.621381 14655 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:31:00.621624 14655 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:31:00.629751 14655 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:31:00.629901 14655 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:31:00.630093 14655 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.18.193
I20260810 11:31:00.640894 14655 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.18.212:34803
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/krb5kdc/oryx_127.8.18.193.keytab
--principal=oryx/127.8.18.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.18.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.18.193
--webserver_port=0
--tserver_master_addrs=127.8.18.254:36523
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.18.193
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:31:00.642474 14655 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:31:00.644030 14655 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:31:00.659668 14661 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:31:00.659745 14660 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:31:00.659766 14663 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:31:00.660823 14655 server_base.cc:1034] running on GCE node
I20260810 11:31:00.661586 14655 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:31:00.662802 14655 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:31:00.664070 14655 hybrid_clock.cc:648] HybridClock initialized: now 1786361460664019 us; error 72 us; skew 500 ppm
I20260810 11:31:00.664440 14655 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:31:00 dist-test-slave-srgv krb5kdc[14545](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361460, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.18.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:31:00.668141 14655 init.cc:377] Logged in from keytab as oryx/127.8.18.193@KRBTEST.COM (short username oryx)
I20260810 11:31:00.669963 14655 webserver.cc:460] Webserver started at http://127.8.18.193:36315/ using document root <none> and password file <none>
I20260810 11:31:00.670806 14655 fs_manager.cc:359] Metadata directory not provided
I20260810 11:31:00.670982 14655 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:31:00.671337 14655 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:31:00.675063 14655 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/ts-0/data/instance:
uuid: "b087d7b7ed0f44b585cbb87fa59cda8b"
format_stamp: "Formatted at 2026-08-10 11:31:00 on dist-test-slave-srgv"
server_key: "2d47d0d5f71a93dd8251d884ae2ed8b8"
server_key_iv: "81d40e980f58f56d6dc4e47e07cf3f82"
server_key_version: "encryptionkey@0"
I20260810 11:31:00.676069 14655 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/ts-0/wal/instance:
uuid: "b087d7b7ed0f44b585cbb87fa59cda8b"
format_stamp: "Formatted at 2026-08-10 11:31:00 on dist-test-slave-srgv"
server_key: "2d47d0d5f71a93dd8251d884ae2ed8b8"
server_key_iv: "81d40e980f58f56d6dc4e47e07cf3f82"
server_key_version: "encryptionkey@0"
I20260810 11:31:00.682339 14655 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.004s
I20260810 11:31:00.686618 14670 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:31:00.688581 14655 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.001s
I20260810 11:31:00.688823 14655 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/ts-0/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/ts-0/wal
uuid: "b087d7b7ed0f44b585cbb87fa59cda8b"
format_stamp: "Formatted at 2026-08-10 11:31:00 on dist-test-slave-srgv"
server_key: "2d47d0d5f71a93dd8251d884ae2ed8b8"
server_key_iv: "81d40e980f58f56d6dc4e47e07cf3f82"
server_key_version: "encryptionkey@0"
I20260810 11:31:00.689091 14655 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:31:00.720353 14655 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:31:00.725858 14655 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:31:00.726224 14655 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:31:00.727350 14655 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:31:00.728976 14655 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:31:00.729099 14655 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:31:00.729264 14655 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:31:00.729352 14655 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:31:00.766930 14655 rpc_server.cc:307] RPC server started. Bound to: 127.8.18.193:44877
I20260810 11:31:00.767000 14787 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.18.193:44877 every 8 connection(s)
I20260810 11:31:00.768514 14655 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/ts-0/data/info.pb
I20260810 11:31:00.776175 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 14655
I20260810 11:31:00.776454 8267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/ts-0/wal/instance
I20260810 11:31:00.777141 8267 external_mini_cluster.cc:1428] Setting key 076dfaffdd30b9f7a87bf2ae8404f292
Aug 10 11:31:00 dist-test-slave-srgv krb5kdc[14545](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361460, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.18.193@KRBTEST.COM for oryx/127.8.18.254@KRBTEST.COM
I20260810 11:31:00.794003 14790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:00.771931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.193:56047 (local address 127.8.18.254:36523)
0810 11:31:00.772520 (+ 589us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:00.772525 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:00.773579 (+ 1054us) server_negotiation.cc:407] Connection header received
0810 11:31:00.775623 (+ 2044us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:00.775629 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:00.775733 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:00.775864 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:31:00.777862 (+ 1998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:00.778850 (+ 988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:00.780782 (+ 1932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:00.781596 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:00.781779 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:00.785981 (+ 4202us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:31:00.785991 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:31:00.785999 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:31:00.786056 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:31:00.789540 (+ 3484us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:31:00.790780 (+ 1240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:31:00.790784 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:31:00.790787 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:31:00.790939 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:31:00.791642 (+ 703us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:31:00.791646 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:31:00.791649 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:31:00.792032 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:31:00.792199 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0810 11:31:00.793220 (+ 1021us) server_negotiation.cc:299] Negotiation successful
0810 11:31:00.793561 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":481,"thread_start_us":352,"threads_started":1}
I20260810 11:31:00.795542 14791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:00.772461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:36523 (local address 127.8.18.193:56047)
0810 11:31:00.773315 (+ 854us) negotiation.cc:107] Waiting for socket to connect
0810 11:31:00.773359 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 11:31:00.775264 (+ 1905us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:31:00.776157 (+ 893us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:31:00.776167 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:31:00.776869 (+ 702us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:31:00.777647 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:00.777669 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:00.779168 (+ 1499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:00.779175 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:00.780576 (+ 1401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:00.780591 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:00.781893 (+ 1302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:00.781900 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:00.782149 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:00.782954 (+ 805us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:31:00.782980 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:31:00.785645 (+ 2665us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:31:00.789760 (+ 4115us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:31:00.789775 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:31:00.789799 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:31:00.790584 (+ 785us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:31:00.791072 (+ 488us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:31:00.791079 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:31:00.791085 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:31:00.791383 (+ 298us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:31:00.792200 (+ 817us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:31:00.792212 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:31:00.792751 (+ 539us) client_negotiation.cc:769] Sending connection context
0810 11:31:00.793085 (+ 334us) client_negotiation.cc:240] Negotiation successful
0810 11:31:00.793874 (+ 789us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":594,"thread_start_us":341,"threads_started":1}
I20260810 11:31:00.797514 14788 heartbeater.cc:347] Connected to a master server at 127.8.18.254:36523
I20260810 11:31:00.797896 14788 heartbeater.cc:464] Registering TS with master...
I20260810 11:31:00.798732 14788 heartbeater.cc:511] Master 127.8.18.254:36523 requested a full tablet report, sending...
I20260810 11:31:00.801235 14591 ts_manager.cc:194] Registered new tserver with Master: b087d7b7ed0f44b585cbb87fa59cda8b (127.8.18.193:44877)
I20260810 11:31:00.803067 14591 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.18.193@KRBTEST.COM'} at 127.8.18.193:56047
WARNING: no policy specified for oryx/127.8.18.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.18.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.18.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/krb5kdc/oryx_127.8.18.194.keytab.
Entry for principal oryx/127.8.18.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/krb5kdc/oryx_127.8.18.194.keytab.
I20260810 11:31:00.838539 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.18.194:0
--local_ip_for_outbound_sockets=127.8.18.194
--webserver_interface=127.8.18.194
--webserver_port=0
--tserver_master_addrs=127.8.18.254:36523
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/krb5kdc/oryx_127.8.18.194.keytab
--principal=oryx/127.8.18.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.18.212:34803
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:31:01.217674 14794 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:31:01.218230 14794 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:31:01.218406 14794 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:31:01.227128 14794 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:31:01.227280 14794 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:31:01.227399 14794 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.18.194
I20260810 11:31:01.238103 14794 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.18.212:34803
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/krb5kdc/oryx_127.8.18.194.keytab
--principal=oryx/127.8.18.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.18.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.18.194
--webserver_port=0
--tserver_master_addrs=127.8.18.254:36523
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.18.194
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:31:01.239550 14794 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:31:01.241029 14794 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:31:01.257390 14799 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:31:01.257355 14800 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:31:01.257443 14802 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:31:01.258514 14794 server_base.cc:1034] running on GCE node
I20260810 11:31:01.259318 14794 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:31:01.260553 14794 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:31:01.261832 14794 hybrid_clock.cc:648] HybridClock initialized: now 1786361461261757 us; error 78 us; skew 500 ppm
I20260810 11:31:01.262234 14794 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:31:01 dist-test-slave-srgv krb5kdc[14545](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361461, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.18.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:31:01.266530 14794 init.cc:377] Logged in from keytab as oryx/127.8.18.194@KRBTEST.COM (short username oryx)
I20260810 11:31:01.268616 14794 webserver.cc:460] Webserver started at http://127.8.18.194:34993/ using document root <none> and password file <none>
I20260810 11:31:01.269671 14794 fs_manager.cc:359] Metadata directory not provided
I20260810 11:31:01.269869 14794 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:31:01.270262 14794 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:31:01.274109 14794 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/ts-1/data/instance:
uuid: "c19c62abc08e457eb6c66d5baff5682d"
format_stamp: "Formatted at 2026-08-10 11:31:01 on dist-test-slave-srgv"
server_key: "bb43c441f10977408bde5045d44813d2"
server_key_iv: "263df37826ac6621e36de7b0cfcb80ba"
server_key_version: "encryptionkey@0"
I20260810 11:31:01.275213 14794 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/ts-1/wal/instance:
uuid: "c19c62abc08e457eb6c66d5baff5682d"
format_stamp: "Formatted at 2026-08-10 11:31:01 on dist-test-slave-srgv"
server_key: "bb43c441f10977408bde5045d44813d2"
server_key_iv: "263df37826ac6621e36de7b0cfcb80ba"
server_key_version: "encryptionkey@0"
I20260810 11:31:01.282159 14794 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260810 11:31:01.286962 14809 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:31:01.288970 14794 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260810 11:31:01.289309 14794 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/ts-1/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/ts-1/wal
uuid: "c19c62abc08e457eb6c66d5baff5682d"
format_stamp: "Formatted at 2026-08-10 11:31:01 on dist-test-slave-srgv"
server_key: "bb43c441f10977408bde5045d44813d2"
server_key_iv: "263df37826ac6621e36de7b0cfcb80ba"
server_key_version: "encryptionkey@0"
I20260810 11:31:01.289638 14794 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:31:01.306679 14794 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:31:01.311986 14794 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:31:01.312363 14794 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:31:01.313609 14794 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:31:01.315228 14794 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:31:01.315351 14794 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:31:01.315514 14794 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:31:01.315601 14794 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:31:01.352360 14794 rpc_server.cc:307] RPC server started. Bound to: 127.8.18.194:40169
I20260810 11:31:01.352408 14925 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.18.194:40169 every 8 connection(s)
I20260810 11:31:01.353953 14794 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/ts-1/data/info.pb
I20260810 11:31:01.356640 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 14794
I20260810 11:31:01.356887 8267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/ts-1/wal/instance
I20260810 11:31:01.357532 8267 external_mini_cluster.cc:1428] Setting key 9169ee6bdb235d6aa1f47a6ffe6239f8
Aug 10 11:31:01 dist-test-slave-srgv krb5kdc[14545](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361461, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.18.194@KRBTEST.COM for oryx/127.8.18.254@KRBTEST.COM
I20260810 11:31:01.381301 14928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:01.357327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.194:45687 (local address 127.8.18.254:36523)
0810 11:31:01.357885 (+ 558us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:01.357889 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:01.359664 (+ 1775us) server_negotiation.cc:407] Connection header received
0810 11:31:01.361839 (+ 2175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:01.361845 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:01.361952 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:01.362114 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:31:01.364072 (+ 1958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:01.365046 (+ 974us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:01.366998 (+ 1952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:01.367612 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:01.367782 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:01.372514 (+ 4732us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:31:01.372524 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:31:01.372532 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:31:01.372589 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:31:01.376672 (+ 4083us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:31:01.377875 (+ 1203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:31:01.377884 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:31:01.377891 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:31:01.378088 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:31:01.378693 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:31:01.378702 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:31:01.378709 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:31:01.379142 (+ 433us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:31:01.379331 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0810 11:31:01.380458 (+ 1127us) server_negotiation.cc:299] Negotiation successful
0810 11:31:01.380789 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":436,"thread_start_us":347,"threads_started":1}
I20260810 11:31:01.382125 14929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:01.358075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:36523 (local address 127.8.18.194:45687)
0810 11:31:01.359427 (+ 1352us) negotiation.cc:107] Waiting for socket to connect
0810 11:31:01.359496 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0810 11:31:01.361398 (+ 1902us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:31:01.362491 (+ 1093us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:31:01.362500 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:31:01.363104 (+ 604us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:31:01.363809 (+ 705us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:01.363831 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:01.365284 (+ 1453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:01.365288 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:01.366776 (+ 1488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:01.366794 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:01.367923 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:01.367929 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:01.368242 (+ 313us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:01.369142 (+ 900us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:31:01.369170 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:31:01.372166 (+ 2996us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:31:01.376946 (+ 4780us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:31:01.376955 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:31:01.376973 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:31:01.377695 (+ 722us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:31:01.378273 (+ 578us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:31:01.378277 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:31:01.378280 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:31:01.378519 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:31:01.379355 (+ 836us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:31:01.379363 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:31:01.380014 (+ 651us) client_negotiation.cc:769] Sending connection context
0810 11:31:01.380281 (+ 267us) client_negotiation.cc:240] Negotiation successful
0810 11:31:01.380774 (+ 493us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1003,"thread_start_us":691,"threads_started":1}
I20260810 11:31:01.384023 14926 heartbeater.cc:347] Connected to a master server at 127.8.18.254:36523
I20260810 11:31:01.384418 14926 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for oryx/127.8.18.195@KRBTEST.COM; defaulting to no policy
I20260810 11:31:01.385380 14926 heartbeater.cc:511] Master 127.8.18.254:36523 requested a full tablet report, sending...
I20260810 11:31:01.387328 14591 ts_manager.cc:194] Registered new tserver with Master: c19c62abc08e457eb6c66d5baff5682d (127.8.18.194:40169)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.18.195@KRBTEST.COM" created.
I20260810 11:31:01.388509 14591 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.18.194@KRBTEST.COM'} at 127.8.18.194:45687
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.18.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/krb5kdc/oryx_127.8.18.195.keytab.
Entry for principal oryx/127.8.18.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/krb5kdc/oryx_127.8.18.195.keytab.
I20260810 11:31:01.421192 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.18.195:0
--local_ip_for_outbound_sockets=127.8.18.195
--webserver_interface=127.8.18.195
--webserver_port=0
--tserver_master_addrs=127.8.18.254:36523
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/krb5kdc/oryx_127.8.18.195.keytab
--principal=oryx/127.8.18.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.18.212:34803
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:31:01.788309 14932 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:31:01.788691 14932 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:31:01.788942 14932 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:31:01.798276 14932 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:31:01.798472 14932 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:31:01.798691 14932 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.18.195
I20260810 11:31:01.807034 14788 heartbeater.cc:503] Master 127.8.18.254:36523 was elected leader, sending a full tablet report...
I20260810 11:31:01.810334 14932 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.18.212:34803
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/krb5kdc/oryx_127.8.18.195.keytab
--principal=oryx/127.8.18.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.18.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.18.195
--webserver_port=0
--tserver_master_addrs=127.8.18.254:36523
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.18.195
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:31:01.811808 14932 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:31:01.813225 14932 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:31:01.829002 14938 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:31:01.829026 14937 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:31:01.828948 14940 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:31:01.829723 14932 server_base.cc:1034] running on GCE node
I20260810 11:31:01.830605 14932 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:31:01.831833 14932 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:31:01.833230 14932 hybrid_clock.cc:648] HybridClock initialized: now 1786361461833171 us; error 63 us; skew 500 ppm
I20260810 11:31:01.833706 14932 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:31:01 dist-test-slave-srgv krb5kdc[14545](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361461, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.18.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:31:01.840988 14932 init.cc:377] Logged in from keytab as oryx/127.8.18.195@KRBTEST.COM (short username oryx)
I20260810 11:31:01.842978 14932 webserver.cc:460] Webserver started at http://127.8.18.195:42243/ using document root <none> and password file <none>
I20260810 11:31:01.843842 14932 fs_manager.cc:359] Metadata directory not provided
I20260810 11:31:01.844034 14932 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:31:01.844406 14932 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:31:01.848012 14932 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/ts-2/data/instance:
uuid: "7f6e17ac59cb474d830f9789a7a7f849"
format_stamp: "Formatted at 2026-08-10 11:31:01 on dist-test-slave-srgv"
server_key: "97c100d0530ccb90f0833cea0839c84d"
server_key_iv: "3d021d5c7c30b69d8dae27784ad8a613"
server_key_version: "encryptionkey@0"
I20260810 11:31:01.848954 14932 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/ts-2/wal/instance:
uuid: "7f6e17ac59cb474d830f9789a7a7f849"
format_stamp: "Formatted at 2026-08-10 11:31:01 on dist-test-slave-srgv"
server_key: "97c100d0530ccb90f0833cea0839c84d"
server_key_iv: "3d021d5c7c30b69d8dae27784ad8a613"
server_key_version: "encryptionkey@0"
I20260810 11:31:01.855624 14932 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.004s
I20260810 11:31:01.860201 14948 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:31:01.861980 14932 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.001s
I20260810 11:31:01.862232 14932 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/ts-2/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/ts-2/wal
uuid: "7f6e17ac59cb474d830f9789a7a7f849"
format_stamp: "Formatted at 2026-08-10 11:31:01 on dist-test-slave-srgv"
server_key: "97c100d0530ccb90f0833cea0839c84d"
server_key_iv: "3d021d5c7c30b69d8dae27784ad8a613"
server_key_version: "encryptionkey@0"
I20260810 11:31:01.862491 14932 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:31:01.876375 14932 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:31:01.881824 14932 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:31:01.882172 14932 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:31:01.883316 14932 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:31:01.884881 14932 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:31:01.885008 14932 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:31:01.885176 14932 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:31:01.885259 14932 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:31:01.923874 14932 rpc_server.cc:307] RPC server started. Bound to: 127.8.18.195:35115
I20260810 11:31:01.923924 15063 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.18.195:35115 every 8 connection(s)
I20260810 11:31:01.925433 14932 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/ts-2/data/info.pb
I20260810 11:31:01.929553 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 14932
I20260810 11:31:01.929795 8267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/ts-2/wal/instance
I20260810 11:31:01.930393 8267 external_mini_cluster.cc:1428] Setting key bdeb2afa7926e1badaa916c02213e267
Aug 10 11:31:01 dist-test-slave-srgv krb5kdc[14545](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361461, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.18.195@KRBTEST.COM for oryx/127.8.18.254@KRBTEST.COM
I20260810 11:31:01.950773 15066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:01.928420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.195:46857 (local address 127.8.18.254:36523)
0810 11:31:01.928960 (+ 540us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:01.928966 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:01.930189 (+ 1223us) server_negotiation.cc:407] Connection header received
0810 11:31:01.932473 (+ 2284us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:01.932478 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:01.932608 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:01.932748 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:31:01.934738 (+ 1990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:01.935542 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:01.937382 (+ 1840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:01.937921 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:01.938096 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:01.942455 (+ 4359us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:31:01.942461 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:31:01.942464 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:31:01.942512 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:31:01.946203 (+ 3691us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:31:01.947404 (+ 1201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:31:01.947410 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:31:01.947414 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:31:01.947573 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:31:01.948257 (+ 684us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:31:01.948262 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:31:01.948266 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:31:01.948780 (+ 514us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:31:01.948998 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0810 11:31:01.950076 (+ 1078us) server_negotiation.cc:299] Negotiation successful
0810 11:31:01.950355 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":412,"thread_start_us":290,"threads_started":1}
I20260810 11:31:01.952175 15067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:01.928881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:36523 (local address 127.8.18.195:46857)
0810 11:31:01.929960 (+ 1079us) negotiation.cc:107] Waiting for socket to connect
0810 11:31:01.930006 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0810 11:31:01.932145 (+ 2139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:31:01.933031 (+ 886us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:31:01.933041 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:31:01.933825 (+ 784us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:31:01.934511 (+ 686us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:01.934532 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:01.935758 (+ 1226us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:01.935762 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:01.937154 (+ 1392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:01.937171 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:01.938228 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:01.938237 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:01.938504 (+ 267us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:01.939356 (+ 852us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:31:01.939404 (+ 48us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:31:01.942102 (+ 2698us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:31:01.946395 (+ 4293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:31:01.946409 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:31:01.946433 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:31:01.947266 (+ 833us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:31:01.947713 (+ 447us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:31:01.947719 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:31:01.947723 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:31:01.948077 (+ 354us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:31:01.948929 (+ 852us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:31:01.948938 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:31:01.949519 (+ 581us) client_negotiation.cc:769] Sending connection context
0810 11:31:01.949921 (+ 402us) client_negotiation.cc:240] Negotiation successful
0810 11:31:01.950541 (+ 620us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":737,"thread_start_us":446,"threads_started":1}
I20260810 11:31:01.954221 15064 heartbeater.cc:347] Connected to a master server at 127.8.18.254:36523
I20260810 11:31:01.954643 15064 heartbeater.cc:464] Registering TS with master...
I20260810 11:31:01.955510 15064 heartbeater.cc:511] Master 127.8.18.254:36523 requested a full tablet report, sending...
I20260810 11:31:01.957317 14591 ts_manager.cc:194] Registered new tserver with Master: 7f6e17ac59cb474d830f9789a7a7f849 (127.8.18.195:35115)
I20260810 11:31:01.958371 14591 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.18.195@KRBTEST.COM'} at 127.8.18.195:46857
I20260810 11:31:01.961127 8267 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.8.18.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.18.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.18.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/krb5kdc/oryx_127.8.18.253.keytab.
Entry for principal oryx/127.8.18.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/krb5kdc/oryx_127.8.18.253.keytab.
I20260810 11:31:02.021911 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.18.253:44341
--webserver_interface=127.8.18.253
--webserver_port=0
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/krb5kdc/oryx_127.8.18.253.keytab
--principal=oryx/127.8.18.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.18.212:34803
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.18.254:36523,127.8.18.253:44341
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 11:31:02.391842 14926 heartbeater.cc:503] Master 127.8.18.254:36523 was elected leader, sending a full tablet report...
W20260810 11:31:02.395852 15070 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:31:02.396214 15070 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:31:02.396376 15070 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:31:02.404253 15070 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 11:31:02.404402 15070 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:31:02.404493 15070 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:31:02.404590 15070 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 11:31:02.404683 15070 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 11:31:02.415678 15070 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.18.212:34803
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.8.18.254:36523,127.8.18.253:44341
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/krb5kdc/oryx_127.8.18.253.keytab
--principal=oryx/127.8.18.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.18.253:44341
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.8.18.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
W20260810 11:31:02.416170 15070 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.8.18.254:36523,127.8.18.253:44341'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260810 11:31:02.417490 15070 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:31:02.418967 15070 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:31:02.432590 15078 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:31:02.432543 15075 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:31:02.432631 15076 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:31:02.434219 15070 server_base.cc:1034] running on GCE node
I20260810 11:31:02.435216 15070 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:31:02.437037 15070 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:31:02.438367 15070 hybrid_clock.cc:648] HybridClock initialized: now 1786361462438289 us; error 76 us; skew 500 ppm
I20260810 11:31:02.438829 15070 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:31:02 dist-test-slave-srgv krb5kdc[14545](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361462, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.18.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:31:02.442757 15070 init.cc:377] Logged in from keytab as oryx/127.8.18.253@KRBTEST.COM (short username oryx)
I20260810 11:31:02.444842 15070 webserver.cc:460] Webserver started at http://127.8.18.253:45015/ using document root <none> and password file <none>
I20260810 11:31:02.445866 15070 fs_manager.cc:359] Metadata directory not provided
I20260810 11:31:02.446039 15070 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:31:02.446516 15070 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:31:02.450230 15070 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/master-1/data/instance:
uuid: "8fb82cee99b449c5a33adb02cba79748"
format_stamp: "Formatted at 2026-08-10 11:31:02 on dist-test-slave-srgv"
server_key: "f09a15d34726a0d63525a8614d5bf87f"
server_key_iv: "f30f593dccff660a37073948f085e99d"
server_key_version: "encryptionkey@0"
I20260810 11:31:02.451157 15070 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/master-1/wal/instance:
uuid: "8fb82cee99b449c5a33adb02cba79748"
format_stamp: "Formatted at 2026-08-10 11:31:02 on dist-test-slave-srgv"
server_key: "f09a15d34726a0d63525a8614d5bf87f"
server_key_iv: "f30f593dccff660a37073948f085e99d"
server_key_version: "encryptionkey@0"
I20260810 11:31:02.457841 15070 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.005s
I20260810 11:31:02.462162 15085 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:31:02.463840 15070 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260810 11:31:02.464169 15070 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/master-1/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/master-1/wal
uuid: "8fb82cee99b449c5a33adb02cba79748"
format_stamp: "Formatted at 2026-08-10 11:31:02 on dist-test-slave-srgv"
server_key: "f09a15d34726a0d63525a8614d5bf87f"
server_key_iv: "f30f593dccff660a37073948f085e99d"
server_key_version: "encryptionkey@0"
I20260810 11:31:02.464459 15070 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:31:02.486533 15070 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:31:02.491863 15070 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:31:02.492174 15070 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:31:02.513538 15070 rpc_server.cc:307] RPC server started. Bound to: 127.8.18.253:44341
I20260810 11:31:02.513566 15136 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.18.253:44341 every 8 connection(s)
I20260810 11:31:02.515157 15070 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/master-1/data/info.pb
I20260810 11:31:02.519619 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 15070
I20260810 11:31:02.519776 15137 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:31:02.530979 15137 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.18.254" port: 36523 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 11:31:02 dist-test-slave-srgv krb5kdc[14545](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361459, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.8.18.253@KRBTEST.COM
Aug 10 11:31:02 dist-test-slave-srgv krb5kdc[14545](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361462, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.18.253@KRBTEST.COM for oryx/127.8.18.254@KRBTEST.COM
I20260810 11:31:02.547211 15140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:02.522078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47910 (local address 127.8.18.253:44341)
0810 11:31:02.523067 (+ 989us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:02.523080 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:02.523138 (+ 58us) server_negotiation.cc:407] Connection header received
0810 11:31:02.525784 (+ 2646us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:02.525816 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:02.526420 (+ 604us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:02.526927 (+ 507us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:31:02.528889 (+ 1962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:02.530486 (+ 1597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:02.531970 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:02.532873 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:02.533116 (+ 243us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:02.538633 (+ 5517us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:31:02.538642 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:31:02.538655 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:31:02.538701 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:31:02.542210 (+ 3509us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:31:02.543058 (+ 848us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:31:02.543066 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:31:02.543075 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:31:02.543293 (+ 218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:31:02.543750 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:31:02.543757 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:31:02.543762 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:31:02.544585 (+ 823us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:31:02.544941 (+ 356us) server_negotiation.cc:1035] Waiting for connection context
0810 11:31:02.545580 (+ 639us) server_negotiation.cc:299] Negotiation successful
0810 11:31:02.546019 (+ 439us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":756,"thread_start_us":469,"threads_started":1}
I20260810 11:31:02.552153 15141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:02.534469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40414 (local address 127.8.18.254:36523)
0810 11:31:02.535160 (+ 691us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:02.535170 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:02.536025 (+ 855us) server_negotiation.cc:407] Connection header received
0810 11:31:02.536252 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:02.536260 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:02.536363 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:02.536506 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:31:02.538180 (+ 1674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:02.538970 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:02.540586 (+ 1616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:02.541013 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:02.541176 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:02.545088 (+ 3912us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:31:02.545096 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:31:02.545102 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:31:02.545165 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:31:02.548718 (+ 3553us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:31:02.549764 (+ 1046us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:31:02.549771 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:31:02.549776 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:31:02.549938 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:31:02.550392 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:31:02.550397 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:31:02.550399 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:31:02.550762 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:31:02.550950 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0810 11:31:02.551465 (+ 515us) server_negotiation.cc:299] Negotiation successful
0810 11:31:02.551736 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":500,"thread_start_us":310,"threads_started":1}
I20260810 11:31:02.552186 15142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:02.534490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:36523 (local address 127.0.0.1:40414)
0810 11:31:02.535822 (+ 1332us) negotiation.cc:107] Waiting for socket to connect
0810 11:31:02.535882 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0810 11:31:02.536092 (+ 210us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:31:02.536645 (+ 553us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:31:02.536655 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:31:02.537671 (+ 1016us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:31:02.537971 (+ 300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:02.537981 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:02.539143 (+ 1162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:02.539151 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:02.540396 (+ 1245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:02.540411 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:02.541232 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:02.541240 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:02.541512 (+ 272us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:02.542244 (+ 732us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:31:02.542263 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:31:02.544831 (+ 2568us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:31:02.548928 (+ 4097us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:31:02.548946 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:31:02.548954 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:31:02.549609 (+ 655us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:31:02.550071 (+ 462us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:31:02.550076 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:31:02.550080 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:31:02.550230 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:31:02.550913 (+ 683us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:31:02.550922 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:31:02.551127 (+ 205us) client_negotiation.cc:769] Sending connection context
0810 11:31:02.551334 (+ 207us) client_negotiation.cc:240] Negotiation successful
0810 11:31:02.551736 (+ 402us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1049,"spinlock_wait_cycles":19200,"thread_start_us":903,"threads_started":1}
I20260810 11:31:02.554694 15137 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.18.253" port: 44341 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 11:31:02 dist-test-slave-srgv krb5kdc[14545](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361462, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.18.253@KRBTEST.COM for oryx/127.8.18.253@KRBTEST.COM
I20260810 11:31:02.570199 15142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:02.555695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.253:44341 (local address 127.0.0.1:47924)
0810 11:31:02.555975 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0810 11:31:02.555997 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 11:31:02.556136 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:31:02.556540 (+ 404us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:31:02.556544 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:31:02.556955 (+ 411us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:31:02.557337 (+ 382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:02.557371 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:02.558535 (+ 1164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:02.558539 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:02.559506 (+ 967us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:02.559522 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:02.560333 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:02.560338 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:02.560484 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:02.561129 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:31:02.561147 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:31:02.563738 (+ 2591us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:31:02.567464 (+ 3726us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:31:02.567470 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:31:02.567473 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:31:02.568022 (+ 549us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:31:02.568490 (+ 468us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:31:02.568494 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:31:02.568497 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:31:02.568644 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:31:02.569354 (+ 710us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:31:02.569359 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:31:02.569555 (+ 196us) client_negotiation.cc:769] Sending connection context
0810 11:31:02.569681 (+ 126us) client_negotiation.cc:240] Negotiation successful
0810 11:31:02.569935 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97}
I20260810 11:31:02.570590 15140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:02.555979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47924 (local address 127.8.18.253:44341)
0810 11:31:02.556167 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:02.556172 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:02.556213 (+ 41us) server_negotiation.cc:407] Connection header received
0810 11:31:02.556271 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:02.556274 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:02.556335 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:02.556436 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:31:02.557574 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:02.558335 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:02.559689 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:02.560126 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:02.560257 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:02.563992 (+ 3735us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:31:02.563997 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:31:02.564001 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:31:02.564052 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:31:02.567264 (+ 3212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:31:02.568184 (+ 920us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:31:02.568189 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:31:02.568191 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:31:02.568316 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:31:02.568747 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:31:02.568751 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:31:02.568753 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:31:02.569154 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:31:02.569301 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0810 11:31:02.569845 (+ 544us) server_negotiation.cc:299] Negotiation successful
0810 11:31:02.570141 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":20,"server-negotiator.queue_time_us":79}
I20260810 11:31:02.575961 15137 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8fb82cee99b449c5a33adb02cba79748: Bootstrap starting.
I20260810 11:31:02.579629 15137 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8fb82cee99b449c5a33adb02cba79748: Neither blocks nor log segments found. Creating new log.
I20260810 11:31:02.580840 15137 log.cc:824] T 00000000000000000000000000000000 P 8fb82cee99b449c5a33adb02cba79748: Log is configured to *not* fsync() on all Append() calls
I20260810 11:31:02.584438 15137 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8fb82cee99b449c5a33adb02cba79748: No bootstrap required, opened a new log
I20260810 11:31:02.593088 15137 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8fb82cee99b449c5a33adb02cba79748 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd8ee5823e444d28a6cb100898b494c5" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 36523 } } peers { permanent_uuid: "8fb82cee99b449c5a33adb02cba79748" member_type: VOTER last_known_addr { host: "127.8.18.253" port: 44341 } }
I20260810 11:31:02.593647 15137 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8fb82cee99b449c5a33adb02cba79748 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:31:02.593792 15137 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8fb82cee99b449c5a33adb02cba79748 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fb82cee99b449c5a33adb02cba79748, State: Initialized, Role: FOLLOWER
I20260810 11:31:02.594658 15137 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8fb82cee99b449c5a33adb02cba79748 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd8ee5823e444d28a6cb100898b494c5" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 36523 } } peers { permanent_uuid: "8fb82cee99b449c5a33adb02cba79748" member_type: VOTER last_known_addr { host: "127.8.18.253" port: 44341 } }
I20260810 11:31:02.595775 15144 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8fb82cee99b449c5a33adb02cba79748 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd8ee5823e444d28a6cb100898b494c5" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 36523 } } peers { permanent_uuid: "8fb82cee99b449c5a33adb02cba79748" member_type: VOTER last_known_addr { host: "127.8.18.253" port: 44341 } } }
I20260810 11:31:02.596165 15144 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8fb82cee99b449c5a33adb02cba79748 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 11:31:02.596998 15137 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8fb82cee99b449c5a33adb02cba79748 [sys.catalog]: configured and running, proceeding with master startup.
W20260810 11:31:02.607439 15155 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8fb82cee99b449c5a33adb02cba79748: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 11:31:02.607635 15155 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 11:31:02.612190 15070 master_runner.cc:428] Detected that this master 8fb82cee99b449c5a33adb02cba79748 is joining an existing cluster
I20260810 11:31:02.612342 15070 master_runner.cc:432] Initiating AddMaster RPC to add 127.8.18.253:44341
I20260810 11:31:02.636596 15141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:02.617378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40422 (local address 127.8.18.254:36523)
0810 11:31:02.617775 (+ 397us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:02.617784 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:02.619110 (+ 1326us) server_negotiation.cc:407] Connection header received
0810 11:31:02.619505 (+ 395us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:02.619512 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:02.619633 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:02.619787 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:31:02.623051 (+ 3264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:02.624209 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:02.626264 (+ 2055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:02.627015 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:02.627102 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:02.629726 (+ 2624us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:31:02.629731 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:31:02.629735 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:31:02.629834 (+ 99us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:31:02.633521 (+ 3687us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:31:02.634325 (+ 804us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:31:02.634331 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:31:02.634335 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:31:02.634483 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:31:02.634934 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:31:02.634939 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:31:02.634943 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:31:02.635259 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:31:02.635449 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0810 11:31:02.635959 (+ 510us) server_negotiation.cc:299] Negotiation successful
0810 11:31:02.636247 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":244}
I20260810 11:31:02.636615 15140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:02.617519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47938 (local address 127.8.18.253:44341)
0810 11:31:02.617816 (+ 297us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:02.617822 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:02.619561 (+ 1739us) server_negotiation.cc:407] Connection header received
0810 11:31:02.619753 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:02.619759 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:02.619824 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:02.619935 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:31:02.621620 (+ 1685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:02.622793 (+ 1173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:02.624557 (+ 1764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:02.625435 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:02.625621 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:02.628629 (+ 3008us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:31:02.628636 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:31:02.628640 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:31:02.628680 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:31:02.632772 (+ 4092us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:31:02.633921 (+ 1149us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:31:02.633927 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:31:02.633930 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:31:02.634113 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:31:02.634573 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:31:02.634578 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:31:02.634582 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:31:02.634981 (+ 399us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:31:02.635155 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0810 11:31:02.635721 (+ 566us) server_negotiation.cc:299] Negotiation successful
0810 11:31:02.636152 (+ 431us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":144,"server-negotiator.queue_time_us":97,"spinlock_wait_cycles":80640}
I20260810 11:31:02.636643 15162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:02.617969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.253:44341 (local address 127.0.0.1:47938)
0810 11:31:02.619456 (+ 1487us) negotiation.cc:107] Waiting for socket to connect
0810 11:31:02.619473 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 11:31:02.619617 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:31:02.620114 (+ 497us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:31:02.620120 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:31:02.620850 (+ 730us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:31:02.621444 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:02.621452 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:02.623029 (+ 1577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:02.623033 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:02.624386 (+ 1353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:02.624405 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:02.625969 (+ 1564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:02.625977 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:02.626182 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:02.627070 (+ 888us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:31:02.627098 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:31:02.628386 (+ 1288us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:31:02.632961 (+ 4575us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:31:02.632968 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:31:02.632972 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:31:02.633773 (+ 801us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:31:02.634255 (+ 482us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:31:02.634260 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:31:02.634265 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:31:02.634447 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:31:02.635226 (+ 779us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:31:02.635231 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:31:02.635390 (+ 159us) client_negotiation.cc:769] Sending connection context
0810 11:31:02.635551 (+ 161us) client_negotiation.cc:240] Negotiation successful
0810 11:31:02.635951 (+ 400us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1341,"mutex_wait_us":42,"thread_start_us":579,"threads_started":1}
I20260810 11:31:02.637765 15161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:02.617519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:36523 (local address 127.0.0.1:40422)
0810 11:31:02.618934 (+ 1415us) negotiation.cc:107] Waiting for socket to connect
0810 11:31:02.618965 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0810 11:31:02.619270 (+ 305us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:31:02.619793 (+ 523us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:31:02.619797 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:31:02.620764 (+ 967us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:31:02.621435 (+ 671us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:02.621450 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:02.624385 (+ 2935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:02.624391 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:02.626025 (+ 1634us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:02.626041 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:02.627369 (+ 1328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:02.627373 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:02.627505 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:02.628139 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:31:02.628156 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:31:02.629449 (+ 1293us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:31:02.633676 (+ 4227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:31:02.633682 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:31:02.633690 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:31:02.634207 (+ 517us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:31:02.634612 (+ 405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:31:02.634617 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:31:02.634620 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:31:02.634780 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:31:02.635393 (+ 613us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:31:02.635398 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:31:02.635630 (+ 232us) client_negotiation.cc:769] Sending connection context
0810 11:31:02.636912 (+ 1282us) client_negotiation.cc:240] Negotiation successful
0810 11:31:02.637246 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1252,"mutex_wait_us":43,"thread_start_us":1560,"threads_started":1}
Aug 10 11:31:02 dist-test-slave-srgv krb5kdc[14545](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361459, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.18.254@KRBTEST.COM for oryx/127.8.18.253@KRBTEST.COM
I20260810 11:31:02.666482 15140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:02.645712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47952 (local address 127.8.18.253:44341)
0810 11:31:02.645952 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:02.645962 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:02.646640 (+ 678us) server_negotiation.cc:407] Connection header received
0810 11:31:02.646972 (+ 332us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:02.646976 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:02.647081 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:02.647213 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:31:02.648962 (+ 1749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:02.649821 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:02.652009 (+ 2188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:02.652528 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:02.652659 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:02.658232 (+ 5573us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:31:02.658240 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:31:02.658246 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:31:02.658293 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:31:02.662741 (+ 4448us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:31:02.663941 (+ 1200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:31:02.663952 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:31:02.663954 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:31:02.664141 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:31:02.664637 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:31:02.664652 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:31:02.664654 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:31:02.664991 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:31:02.665167 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0810 11:31:02.665780 (+ 613us) server_negotiation.cc:299] Negotiation successful
0810 11:31:02.666053 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260810 11:31:02.666618 15163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:02.645690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.253:44341 (local address 127.0.0.1:47952)
0810 11:31:02.646448 (+ 758us) negotiation.cc:107] Waiting for socket to connect
0810 11:31:02.646494 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0810 11:31:02.646750 (+ 256us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:31:02.647281 (+ 531us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:31:02.647292 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:31:02.648362 (+ 1070us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:31:02.648727 (+ 365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:02.648746 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:02.649999 (+ 1253us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:02.650006 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:02.651791 (+ 1785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:02.651815 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:02.652755 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:02.652763 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:02.653357 (+ 594us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:02.654301 (+ 944us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:31:02.654334 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:31:02.657931 (+ 3597us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:31:02.662995 (+ 5064us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:31:02.663020 (+ 25us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:31:02.663031 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:31:02.663756 (+ 725us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:31:02.664294 (+ 538us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:31:02.664299 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:31:02.664303 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:31:02.664497 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:31:02.665143 (+ 646us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:31:02.665161 (+ 18us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:31:02.665362 (+ 201us) client_negotiation.cc:769] Sending connection context
0810 11:31:02.665655 (+ 293us) client_negotiation.cc:240] Negotiation successful
0810 11:31:02.666114 (+ 459us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":478,"thread_start_us":305,"threads_started":1}
I20260810 11:31:02.669210 14591 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.8.18.253:44341: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "8fb82cee99b449c5a33adb02cba79748"
member_type: NON_VOTER
last_known_addr {
host: "127.8.18.253"
port: 44341
}
attrs {
promote: true
}
}
dest_uuid: "cd8ee5823e444d28a6cb100898b494c5"
cas_config_opid_index: -1
I20260810 11:31:02.670830 14591 consensus_queue.cc:237] T 00000000000000000000000000000000 P cd8ee5823e444d28a6cb100898b494c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "cd8ee5823e444d28a6cb100898b494c5" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 36523 } } peers { permanent_uuid: "8fb82cee99b449c5a33adb02cba79748" member_type: NON_VOTER last_known_addr { host: "127.8.18.253" port: 44341 } attrs { promote: true } }
WARNING: no policy specified for oryx/127.8.18.252@KRBTEST.COM; defaulting to no policy
I20260810 11:31:02.684227 15168 raft_consensus.cc:2932] T 00000000000000000000000000000000 P cd8ee5823e444d28a6cb100898b494c5 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 8fb82cee99b449c5a33adb02cba79748 (127.8.18.253) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "cd8ee5823e444d28a6cb100898b494c5" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 36523 } } peers { permanent_uuid: "8fb82cee99b449c5a33adb02cba79748" member_type: NON_VOTER last_known_addr { host: "127.8.18.253" port: 44341 } attrs { promote: true } } }
I20260810 11:31:02.685952 15112 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8fb82cee99b449c5a33adb02cba79748 [term 0 FOLLOWER]: Advancing to term 1
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.18.252@KRBTEST.COM" created.
I20260810 11:31:02.689904 15168 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.8.18.253:44341
I20260810 11:31:02.690591 15112 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8fb82cee99b449c5a33adb02cba79748 [term 1 FOLLOWER]: Refusing update from remote peer cd8ee5823e444d28a6cb100898b494c5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260810 11:31:02.690861 15168 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd8ee5823e444d28a6cb100898b494c5 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "cd8ee5823e444d28a6cb100898b494c5" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "cd8ee5823e444d28a6cb100898b494c5" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 36523 } } peers { permanent_uuid: "8fb82cee99b449c5a33adb02cba79748" member_type: NON_VOTER last_known_addr { host: "127.8.18.253" port: 44341 } attrs { promote: true } } }
I20260810 11:31:02.691203 15070 master.cc:537] Master@127.8.18.253:44341 shutting down...
I20260810 11:31:02.691223 15168 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd8ee5823e444d28a6cb100898b494c5 [sys.catalog]: This master's current role is: LEADER
I20260810 11:31:02.693334 15168 consensus_queue.cc:1035] T 00000000000000000000000000000000 P cd8ee5823e444d28a6cb100898b494c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fb82cee99b449c5a33adb02cba79748" member_type: NON_VOTER last_known_addr { host: "127.8.18.253" port: 44341 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260810 11:31:02.697968 14576 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260810 11:31:02.705149 14576 consensus_peers.cc:487] T 00000000000000000000000000000000 P cd8ee5823e444d28a6cb100898b494c5 -> Peer 8fb82cee99b449c5a33adb02cba79748 (127.8.18.253:44341): Couldn't send request to peer 8fb82cee99b449c5a33adb02cba79748. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260810 11:31:02.710736 15070 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 8fb82cee99b449c5a33adb02cba79748 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 11:31:02.711403 15070 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 8fb82cee99b449c5a33adb02cba79748 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:31:02.711596 15070 tablet_replica.cc:330] T 00000000000000000000000000000000 P 8fb82cee99b449c5a33adb02cba79748: stopping tablet replica
I20260810 11:31:02.719717 15070 master.cc:552] Master@127.8.18.253:44341 shutdown complete.
I20260810 11:31:02.719892 15070 master_runner.cc:305] Clearing existing system tablet
I20260810 11:31:02.720466 15070 fs_manager.cc:359] Metadata directory not provided
I20260810 11:31:02.720607 15070 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:31:02.724304 15070 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260810 11:31:02.726804 15173 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:31:02.727277 15070 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:31:02.727592 15070 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 11:31:02.727687 15070 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/master-1/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/master-1/wal
uuid: "8fb82cee99b449c5a33adb02cba79748"
format_stamp: "Formatted at 2026-08-10 11:31:02 on dist-test-slave-srgv"
server_key: "f09a15d34726a0d63525a8614d5bf87f"
server_key_iv: "f30f593dccff660a37073948f085e99d"
server_key_version: "encryptionkey@0"
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.18.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/krb5kdc/oryx_127.8.18.252.keytab.
Entry for principal oryx/127.8.18.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/krb5kdc/oryx_127.8.18.252.keytab.
I20260810 11:31:02.733271 15070 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 8fb82cee99b449c5a33adb02cba79748: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:31:02.734220 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.18.252:41545
--webserver_interface=127.8.18.252
--webserver_port=0
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/krb5kdc/oryx_127.8.18.252.keytab
--principal=oryx/127.8.18.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.18.212:34803
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.18.254:36523,127.8.18.253:44341,127.8.18.252:41545
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 11:31:02.739024 15070 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 8fb82cee99b449c5a33adb02cba79748: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260810 11:31:02.739329 15070 log.cc:1196] T 00000000000000000000000000000000 P 8fb82cee99b449c5a33adb02cba79748: Deleting WAL directory at /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260810 11:31:02.739816 15070 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 8fb82cee99b449c5a33adb02cba79748: Deleting consensus metadata
I20260810 11:31:02.740132 15070 master_runner.cc:315] Copying system tablet from 127.8.18.254:36523
I20260810 11:31:02.747318 15070 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 8fb82cee99b449c5a33adb02cba79748: tablet copy: Beginning tablet copy session from remote peer at address 127.8.18.254:36523
I20260810 11:31:02.768569 15181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:02.748493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:36523 (local address 127.0.0.1:40438)
0810 11:31:02.749978 (+ 1485us) negotiation.cc:107] Waiting for socket to connect
0810 11:31:02.750005 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0810 11:31:02.750303 (+ 298us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:31:02.750797 (+ 494us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:31:02.750804 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:31:02.751472 (+ 668us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:31:02.751817 (+ 345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:02.751837 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:02.753483 (+ 1646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:02.753491 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:02.755346 (+ 1855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:02.755373 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:02.756834 (+ 1461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:02.756841 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:02.757057 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:02.757954 (+ 897us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:31:02.757980 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:31:02.759261 (+ 1281us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:31:02.764593 (+ 5332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:31:02.764601 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:31:02.764606 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:31:02.765236 (+ 630us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:31:02.765886 (+ 650us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:31:02.765894 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:31:02.765898 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:31:02.766131 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:31:02.767010 (+ 879us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:31:02.767015 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:31:02.767264 (+ 249us) client_negotiation.cc:769] Sending connection context
0810 11:31:02.767637 (+ 373us) client_negotiation.cc:240] Negotiation successful
0810 11:31:02.768028 (+ 391us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1268,"thread_start_us":1137,"threads_started":1}
I20260810 11:31:02.769948 15141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:02.748713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40438 (local address 127.8.18.254:36523)
0810 11:31:02.748944 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:02.748953 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:02.750132 (+ 1179us) server_negotiation.cc:407] Connection header received
0810 11:31:02.750512 (+ 380us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:02.750518 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:02.750611 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:02.750733 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:31:02.752035 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:02.753230 (+ 1195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:02.755592 (+ 2362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:02.756375 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:02.756534 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:02.759604 (+ 3070us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:31:02.759614 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:31:02.759622 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:31:02.759685 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:31:02.764332 (+ 4647us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:31:02.765514 (+ 1182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:31:02.765522 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:31:02.765526 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:31:02.765727 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:31:02.766322 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:31:02.766330 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:31:02.766335 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:31:02.766770 (+ 435us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:31:02.766937 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0810 11:31:02.769224 (+ 2287us) server_negotiation.cc:299] Negotiation successful
0810 11:31:02.769560 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260810 11:31:02.771076 14612 tablet_copy_service.cc:140] P cd8ee5823e444d28a6cb100898b494c5: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 8fb82cee99b449c5a33adb02cba79748 ({username='oryx', principal='oryx/127.8.18.253@KRBTEST.COM'} at 127.0.0.1:40438)
I20260810 11:31:02.771452 14612 tablet_copy_service.cc:161] P cd8ee5823e444d28a6cb100898b494c5: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 8fb82cee99b449c5a33adb02cba79748 at {username='oryx', principal='oryx/127.8.18.253@KRBTEST.COM'} at 127.0.0.1:40438: session id = 8fb82cee99b449c5a33adb02cba79748-00000000000000000000000000000000
I20260810 11:31:02.777066 14612 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P cd8ee5823e444d28a6cb100898b494c5: Tablet Copy: opened 0 blocks and 1 log segments
I20260810 11:31:02.780416 15070 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260810 11:31:02.788141 15070 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 8fb82cee99b449c5a33adb02cba79748: tablet copy: Starting download of 0 data blocks...
I20260810 11:31:02.788632 15070 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 8fb82cee99b449c5a33adb02cba79748: tablet copy: Starting download of 1 WAL segments...
I20260810 11:31:02.792907 15070 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 8fb82cee99b449c5a33adb02cba79748: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260810 11:31:02.798882 14612 tablet_copy_service.cc:342] P cd8ee5823e444d28a6cb100898b494c5: Request end of tablet copy session 8fb82cee99b449c5a33adb02cba79748-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.8.18.253@KRBTEST.COM'} at 127.0.0.1:40438
I20260810 11:31:02.799285 14612 tablet_copy_service.cc:433] P cd8ee5823e444d28a6cb100898b494c5: ending tablet copy session 8fb82cee99b449c5a33adb02cba79748-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 8fb82cee99b449c5a33adb02cba79748
I20260810 11:31:02.815769 15070 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:31:02.818796 15182 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:31:02.820209 15183 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:31:02.828691 15185 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:31:02.828929 15070 server_base.cc:1034] running on GCE node
I20260810 11:31:02.829560 15070 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:31:02.830384 15070 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:31:02.831773 15070 hybrid_clock.cc:648] HybridClock initialized: now 1786361462831710 us; error 43 us; skew 500 ppm
I20260810 11:31:02.832129 15070 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:31:02 dist-test-slave-srgv krb5kdc[14545](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361462, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.18.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:31:02.834784 15070 init.cc:377] Logged in from keytab as oryx/127.8.18.253@KRBTEST.COM (short username oryx)
I20260810 11:31:02.836438 15070 webserver.cc:460] Webserver started at http://127.8.18.253:40231/ using document root <none> and password file <none>
I20260810 11:31:02.836956 15070 fs_manager.cc:359] Metadata directory not provided
I20260810 11:31:02.837121 15070 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:31:02.840960 15070 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260810 11:31:02.843611 15193 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:31:02.844235 15070 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260810 11:31:02.844460 15070 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/master-1/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/master-1/wal
uuid: "8fb82cee99b449c5a33adb02cba79748"
format_stamp: "Formatted at 2026-08-10 11:31:02 on dist-test-slave-srgv"
server_key: "f09a15d34726a0d63525a8614d5bf87f"
server_key_iv: "f30f593dccff660a37073948f085e99d"
server_key_version: "encryptionkey@0"
I20260810 11:31:02.844731 15070 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:31:02.897138 15070 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:31:02.901980 15070 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:31:02.941329 15070 rpc_server.cc:307] RPC server started. Bound to: 127.8.18.253:44341
I20260810 11:31:02.941358 15244 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.18.253:44341 every 8 connection(s)
I20260810 11:31:02.942643 15070 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/master-1/data/info.pb
I20260810 11:31:02.951536 15245 sys_catalog.cc:263] Verifying existing consensus state
I20260810 11:31:02.959939 15245 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8fb82cee99b449c5a33adb02cba79748: Bootstrap starting.
I20260810 11:31:02.962028 15064 heartbeater.cc:503] Master 127.8.18.254:36523 was elected leader, sending a full tablet report...
I20260810 11:31:02.993311 15245 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8fb82cee99b449c5a33adb02cba79748: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 11:31:02.994117 15245 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8fb82cee99b449c5a33adb02cba79748: Bootstrap complete.
I20260810 11:31:02.995743 15245 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8fb82cee99b449c5a33adb02cba79748 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "cd8ee5823e444d28a6cb100898b494c5" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 36523 } } peers { permanent_uuid: "8fb82cee99b449c5a33adb02cba79748" member_type: NON_VOTER last_known_addr { host: "127.8.18.253" port: 44341 } attrs { promote: true } }
I20260810 11:31:02.996188 15245 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8fb82cee99b449c5a33adb02cba79748 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 8fb82cee99b449c5a33adb02cba79748, State: Initialized, Role: LEARNER
I20260810 11:31:02.996529 15245 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8fb82cee99b449c5a33adb02cba79748 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "cd8ee5823e444d28a6cb100898b494c5" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 36523 } } peers { permanent_uuid: "8fb82cee99b449c5a33adb02cba79748" member_type: NON_VOTER last_known_addr { host: "127.8.18.253" port: 44341 } attrs { promote: true } }
I20260810 11:31:02.997722 15248 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8fb82cee99b449c5a33adb02cba79748 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "cd8ee5823e444d28a6cb100898b494c5" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 36523 } } peers { permanent_uuid: "8fb82cee99b449c5a33adb02cba79748" member_type: NON_VOTER last_known_addr { host: "127.8.18.253" port: 44341 } attrs { promote: true } } }
I20260810 11:31:02.998133 15248 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8fb82cee99b449c5a33adb02cba79748 [sys.catalog]: This master's current role is: LEARNER
I20260810 11:31:02.998167 15245 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8fb82cee99b449c5a33adb02cba79748 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 11:31:03.031322 15259 catalog_manager.cc:1259] Loaded cluster ID: 63349ed0186a4282894be80560e2d468
I20260810 11:31:03.031478 15259 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 8fb82cee99b449c5a33adb02cba79748: loading cluster ID for follower catalog manager: success
I20260810 11:31:03.035917 15259 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 8fb82cee99b449c5a33adb02cba79748: acquiring CA information for follower catalog manager: success
I20260810 11:31:03.039819 15259 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 8fb82cee99b449c5a33adb02cba79748: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 11:31:03.256109 15264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:03.247224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.253:44341 (local address 127.0.0.1:47954)
0810 11:31:03.247812 (+ 588us) negotiation.cc:107] Waiting for socket to connect
0810 11:31:03.247841 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0810 11:31:03.248092 (+ 251us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:31:03.248968 (+ 876us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:31:03.248971 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:31:03.248999 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:31:03.249248 (+ 249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:03.249257 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:03.251797 (+ 2540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:03.251803 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:03.253233 (+ 1430us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:03.253253 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:03.255368 (+ 2115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:03.255374 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:03.255532 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:03.255587 (+ 55us) client_negotiation.cc:769] Sending connection context
0810 11:31:03.255705 (+ 118us) client_negotiation.cc:240] Negotiation successful
0810 11:31:03.255827 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":413,"thread_start_us":286,"threads_started":1}
I20260810 11:31:03.256546 15265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:03.247481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47954 (local address 127.8.18.253:44341)
0810 11:31:03.248445 (+ 964us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:03.248452 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:03.248475 (+ 23us) server_negotiation.cc:407] Connection header received
0810 11:31:03.248554 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:03.248559 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:03.248709 (+ 150us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:03.248895 (+ 186us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:31:03.249504 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:03.251530 (+ 2026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:03.253496 (+ 1966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:03.255001 (+ 1505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:03.255195 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:03.255499 (+ 304us) server_negotiation.cc:1035] Waiting for connection context
0810 11:31:03.255768 (+ 269us) server_negotiation.cc:299] Negotiation successful
0810 11:31:03.255928 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":783,"thread_start_us":555,"threads_started":1}
I20260810 11:31:03.257860 15220 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 8fb82cee99b449c5a33adb02cba79748 [term 1 LEARNER]: Deduplicated request from leader. Original: 0.0->[1.1-1.5] Dedup: 1.5->[]
W20260810 11:31:03.258540 15178 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
I20260810 11:31:03.258693 15248 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8fb82cee99b449c5a33adb02cba79748 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cd8ee5823e444d28a6cb100898b494c5. Latest consensus state: current_term: 1 leader_uuid: "cd8ee5823e444d28a6cb100898b494c5" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "cd8ee5823e444d28a6cb100898b494c5" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 36523 } } peers { permanent_uuid: "8fb82cee99b449c5a33adb02cba79748" member_type: NON_VOTER last_known_addr { host: "127.8.18.253" port: 44341 } attrs { promote: true } } }
I20260810 11:31:03.259040 15248 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8fb82cee99b449c5a33adb02cba79748 [sys.catalog]: This master's current role is: LEARNER
W20260810 11:31:03.259109 15178 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:31:03.259327 15178 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:31:03.269655 15178 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 11:31:03.269862 15178 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:31:03.269980 15178 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:31:03.270115 15178 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 11:31:03.270262 15178 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 11:31:03.283334 15178 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.18.212:34803
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.8.18.254:36523,127.8.18.253:44341,127.8.18.252:41545
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/krb5kdc/oryx_127.8.18.252.keytab
--principal=oryx/127.8.18.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.18.252:41545
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.8.18.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:31:03.285085 15178 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:31:03.286695 15178 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:31:03.305351 15267 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:31:03.305213 15268 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:31:03.305127 15270 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:31:03.307142 15178 server_base.cc:1034] running on GCE node
I20260810 11:31:03.308665 15178 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:31:03.311110 15178 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:31:03.312505 15178 hybrid_clock.cc:648] HybridClock initialized: now 1786361463312367 us; error 76 us; skew 500 ppm
I20260810 11:31:03.313091 15178 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:31:03 dist-test-slave-srgv krb5kdc[14545](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361463, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.18.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:31:03.318107 15178 init.cc:377] Logged in from keytab as oryx/127.8.18.252@KRBTEST.COM (short username oryx)
I20260810 11:31:03.320202 15178 webserver.cc:460] Webserver started at http://127.8.18.252:37235/ using document root <none> and password file <none>
I20260810 11:31:03.321125 15178 fs_manager.cc:359] Metadata directory not provided
I20260810 11:31:03.321326 15178 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:31:03.321776 15178 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:31:03.326139 15178 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/master-2/data/instance:
uuid: "d1ea41131c2a4c6f984571036be55232"
format_stamp: "Formatted at 2026-08-10 11:31:03 on dist-test-slave-srgv"
server_key: "7c97d9bd2cf22ac2d531541b5cf22a8d"
server_key_iv: "767195ad4a0e0f0255eda1652ac571c1"
server_key_version: "encryptionkey@0"
I20260810 11:31:03.327265 15178 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/master-2/wal/instance:
uuid: "d1ea41131c2a4c6f984571036be55232"
format_stamp: "Formatted at 2026-08-10 11:31:03 on dist-test-slave-srgv"
server_key: "7c97d9bd2cf22ac2d531541b5cf22a8d"
server_key_iv: "767195ad4a0e0f0255eda1652ac571c1"
server_key_version: "encryptionkey@0"
I20260810 11:31:03.334678 15178 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.007s sys 0.001s
I20260810 11:31:03.339681 15277 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:31:03.341641 15178 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260810 11:31:03.341913 15178 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/master-2/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/master-2/wal
uuid: "d1ea41131c2a4c6f984571036be55232"
format_stamp: "Formatted at 2026-08-10 11:31:03 on dist-test-slave-srgv"
server_key: "7c97d9bd2cf22ac2d531541b5cf22a8d"
server_key_iv: "767195ad4a0e0f0255eda1652ac571c1"
server_key_version: "encryptionkey@0"
I20260810 11:31:03.342252 15178 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:31:03.372452 15178 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:31:03.378183 15178 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:31:03.378582 15178 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:31:03.411222 15178 rpc_server.cc:307] RPC server started. Bound to: 127.8.18.252:41545
I20260810 11:31:03.411271 15329 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.18.252:41545 every 8 connection(s)
I20260810 11:31:03.412995 15178 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/master-2/data/info.pb
I20260810 11:31:03.417222 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 15178
I20260810 11:31:03.418174 15330 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:31:03.429759 15330 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.18.254" port: 36523 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 11:31:03 dist-test-slave-srgv krb5kdc[14545](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361459, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.8.18.252@KRBTEST.COM
Aug 10 11:31:03 dist-test-slave-srgv krb5kdc[14545](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361463, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.18.252@KRBTEST.COM for oryx/127.8.18.254@KRBTEST.COM
I20260810 11:31:03.450649 15334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:03.419714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45080 (local address 127.8.18.252:41545)
0810 11:31:03.421031 (+ 1317us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:03.421129 (+ 98us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:03.421587 (+ 458us) server_negotiation.cc:407] Connection header received
0810 11:31:03.424393 (+ 2806us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:03.424424 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:03.425022 (+ 598us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:03.425621 (+ 599us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:31:03.427554 (+ 1933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:03.429276 (+ 1722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:03.430914 (+ 1638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:03.431738 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:03.431988 (+ 250us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:03.438774 (+ 6786us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:31:03.438791 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:31:03.438811 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:31:03.438883 (+ 72us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:31:03.444221 (+ 5338us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:31:03.445544 (+ 1323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:31:03.445555 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:31:03.445567 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:31:03.445898 (+ 331us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:31:03.446623 (+ 725us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:31:03.446628 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:31:03.446630 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:31:03.447466 (+ 836us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:31:03.447932 (+ 466us) server_negotiation.cc:1035] Waiting for connection context
0810 11:31:03.448663 (+ 731us) server_negotiation.cc:299] Negotiation successful
0810 11:31:03.449353 (+ 690us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1003,"thread_start_us":659,"threads_started":1}
I20260810 11:31:03.454342 15336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:03.432828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:36523 (local address 127.0.0.1:40446)
0810 11:31:03.434283 (+ 1455us) negotiation.cc:107] Waiting for socket to connect
0810 11:31:03.434351 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0810 11:31:03.434641 (+ 290us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:31:03.435613 (+ 972us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:31:03.435622 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:31:03.436602 (+ 980us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:31:03.437200 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:03.437220 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:03.438715 (+ 1495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:03.438722 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:03.440983 (+ 2261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:03.441008 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:03.442292 (+ 1284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:03.442302 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:03.442616 (+ 314us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:03.443389 (+ 773us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:31:03.443421 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:31:03.446754 (+ 3333us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:31:03.451065 (+ 4311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:31:03.451090 (+ 25us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:31:03.451099 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:31:03.451735 (+ 636us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:31:03.452272 (+ 537us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:31:03.452277 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:31:03.452281 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:31:03.452462 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:31:03.453078 (+ 616us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:31:03.453086 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:31:03.453292 (+ 206us) client_negotiation.cc:769] Sending connection context
0810 11:31:03.453616 (+ 324us) client_negotiation.cc:240] Negotiation successful
0810 11:31:03.453997 (+ 381us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":985,"thread_start_us":696,"threads_started":1}
I20260810 11:31:03.454358 15335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:03.432946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40446 (local address 127.8.18.254:36523)
0810 11:31:03.435081 (+ 2135us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:03.435089 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:03.435117 (+ 28us) server_negotiation.cc:407] Connection header received
0810 11:31:03.435194 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:03.435201 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:03.435328 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:03.435513 (+ 185us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:31:03.437507 (+ 1994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:03.438522 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:03.441285 (+ 2763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:03.441947 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:03.442068 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:03.446979 (+ 4911us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:31:03.446985 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:31:03.446988 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:31:03.447029 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:31:03.450893 (+ 3864us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:31:03.451893 (+ 1000us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:31:03.451902 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:31:03.451906 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:31:03.452120 (+ 214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:31:03.452591 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:31:03.452597 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:31:03.452600 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:31:03.452900 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:31:03.453089 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0810 11:31:03.453746 (+ 657us) server_negotiation.cc:299] Negotiation successful
0810 11:31:03.453996 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1977,"thread_start_us":1838,"threads_started":1}
I20260810 11:31:03.455978 15330 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.18.253" port: 44341 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 11:31:03 dist-test-slave-srgv krb5kdc[14545](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361463, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.18.252@KRBTEST.COM for oryx/127.8.18.253@KRBTEST.COM
I20260810 11:31:03.474228 15336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:03.457190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.253:44341 (local address 127.0.0.1:47964)
0810 11:31:03.457540 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0810 11:31:03.457567 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0810 11:31:03.457720 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:31:03.458129 (+ 409us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:31:03.458134 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:31:03.458644 (+ 510us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:31:03.458814 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:03.458823 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:03.460083 (+ 1260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:03.460097 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:03.461216 (+ 1119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:03.461232 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:03.462246 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:03.462253 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:03.462522 (+ 269us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:03.463363 (+ 841us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:31:03.463384 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:31:03.466410 (+ 3026us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:31:03.470857 (+ 4447us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:31:03.470863 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:31:03.470867 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:31:03.471480 (+ 613us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:31:03.472047 (+ 567us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:31:03.472053 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:31:03.472057 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:31:03.472272 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:31:03.472950 (+ 678us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:31:03.472955 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:31:03.473137 (+ 182us) client_negotiation.cc:769] Sending connection context
0810 11:31:03.473294 (+ 157us) client_negotiation.cc:240] Negotiation successful
0810 11:31:03.473764 (+ 470us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100}
I20260810 11:31:03.474303 15265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:03.457347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47964 (local address 127.8.18.253:44341)
0810 11:31:03.457672 (+ 325us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:03.457679 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:03.457702 (+ 23us) server_negotiation.cc:407] Connection header received
0810 11:31:03.457900 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:03.457905 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:03.457974 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:03.458098 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:31:03.458990 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:03.459891 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:03.461461 (+ 1570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:03.462015 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:03.462186 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:03.466734 (+ 4548us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:31:03.466740 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:31:03.466744 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:31:03.466785 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:31:03.470646 (+ 3861us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:31:03.471637 (+ 991us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:31:03.471643 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:31:03.471647 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:31:03.471901 (+ 254us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:31:03.472402 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:31:03.472409 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:31:03.472413 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:31:03.472813 (+ 400us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:31:03.472990 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0810 11:31:03.473437 (+ 447us) server_negotiation.cc:299] Negotiation successful
0810 11:31:03.473763 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133}
I20260810 11:31:03.475919 15330 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.18.252" port: 41545 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 11:31:03 dist-test-slave-srgv krb5kdc[14545](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361463, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.18.252@KRBTEST.COM for oryx/127.8.18.252@KRBTEST.COM
I20260810 11:31:03.491621 15336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:03.476624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.252:41545 (local address 127.0.0.1:45096)
0810 11:31:03.476870 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0810 11:31:03.476887 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 11:31:03.476974 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:31:03.477282 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:31:03.477285 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:31:03.477795 (+ 510us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:31:03.478003 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:03.478012 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:03.479166 (+ 1154us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:03.479170 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:03.480290 (+ 1120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:03.480305 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:03.481183 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:03.481188 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:03.481542 (+ 354us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:03.482208 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:31:03.482227 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:31:03.484830 (+ 2603us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:31:03.488632 (+ 3802us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:31:03.488637 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:31:03.488642 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:31:03.489099 (+ 457us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:31:03.489587 (+ 488us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:31:03.489593 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:31:03.489597 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:31:03.489795 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:31:03.490376 (+ 581us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:31:03.490381 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:31:03.490546 (+ 165us) client_negotiation.cc:769] Sending connection context
0810 11:31:03.490696 (+ 150us) client_negotiation.cc:240] Negotiation successful
0810 11:31:03.490999 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":66}
I20260810 11:31:03.491636 15334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:03.476738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45096 (local address 127.8.18.252:41545)
0810 11:31:03.476900 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:03.476904 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:03.476944 (+ 40us) server_negotiation.cc:407] Connection header received
0810 11:31:03.477086 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:03.477089 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:03.477153 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:03.477241 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:31:03.478174 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:03.478995 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:03.480456 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:03.480919 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:03.481070 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:03.485057 (+ 3987us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:31:03.485063 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:31:03.485066 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:31:03.485099 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:31:03.488447 (+ 3348us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:31:03.489237 (+ 790us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:31:03.489244 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:31:03.489247 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:31:03.489469 (+ 222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:31:03.489906 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:31:03.489914 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:31:03.489917 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:31:03.490238 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:31:03.490395 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0810 11:31:03.490790 (+ 395us) server_negotiation.cc:299] Negotiation successful
0810 11:31:03.491172 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":143,"server-negotiator.queue_time_us":61}
I20260810 11:31:03.498453 15330 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d1ea41131c2a4c6f984571036be55232: Bootstrap starting.
I20260810 11:31:03.502529 15330 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d1ea41131c2a4c6f984571036be55232: Neither blocks nor log segments found. Creating new log.
I20260810 11:31:03.503930 15330 log.cc:824] T 00000000000000000000000000000000 P d1ea41131c2a4c6f984571036be55232: Log is configured to *not* fsync() on all Append() calls
I20260810 11:31:03.508024 15330 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d1ea41131c2a4c6f984571036be55232: No bootstrap required, opened a new log
I20260810 11:31:03.517347 15330 raft_consensus.cc:348] T 00000000000000000000000000000000 P d1ea41131c2a4c6f984571036be55232 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd8ee5823e444d28a6cb100898b494c5" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 36523 } } peers { permanent_uuid: "8fb82cee99b449c5a33adb02cba79748" member_type: VOTER last_known_addr { host: "127.8.18.253" port: 44341 } } peers { permanent_uuid: "d1ea41131c2a4c6f984571036be55232" member_type: VOTER last_known_addr { host: "127.8.18.252" port: 41545 } }
I20260810 11:31:03.518013 15330 raft_consensus.cc:374] T 00000000000000000000000000000000 P d1ea41131c2a4c6f984571036be55232 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:31:03.518262 15330 raft_consensus.cc:729] T 00000000000000000000000000000000 P d1ea41131c2a4c6f984571036be55232 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1ea41131c2a4c6f984571036be55232, State: Initialized, Role: FOLLOWER
I20260810 11:31:03.519025 15330 consensus_queue.cc:260] T 00000000000000000000000000000000 P d1ea41131c2a4c6f984571036be55232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd8ee5823e444d28a6cb100898b494c5" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 36523 } } peers { permanent_uuid: "8fb82cee99b449c5a33adb02cba79748" member_type: VOTER last_known_addr { host: "127.8.18.253" port: 44341 } } peers { permanent_uuid: "d1ea41131c2a4c6f984571036be55232" member_type: VOTER last_known_addr { host: "127.8.18.252" port: 41545 } }
I20260810 11:31:03.520732 15338 sys_catalog.cc:455] T 00000000000000000000000000000000 P d1ea41131c2a4c6f984571036be55232 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd8ee5823e444d28a6cb100898b494c5" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 36523 } } peers { permanent_uuid: "8fb82cee99b449c5a33adb02cba79748" member_type: VOTER last_known_addr { host: "127.8.18.253" port: 44341 } } peers { permanent_uuid: "d1ea41131c2a4c6f984571036be55232" member_type: VOTER last_known_addr { host: "127.8.18.252" port: 41545 } } }
I20260810 11:31:03.521541 15338 sys_catalog.cc:458] T 00000000000000000000000000000000 P d1ea41131c2a4c6f984571036be55232 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 11:31:03.522164 15330 sys_catalog.cc:564] T 00000000000000000000000000000000 P d1ea41131c2a4c6f984571036be55232 [sys.catalog]: configured and running, proceeding with master startup.
W20260810 11:31:03.554885 15349 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d1ea41131c2a4c6f984571036be55232: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 11:31:03.555239 15349 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 11:31:03.558996 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 14655
I20260810 11:31:03.560822 15178 master_runner.cc:428] Detected that this master d1ea41131c2a4c6f984571036be55232 is joining an existing cluster
I20260810 11:31:03.561129 15178 master_runner.cc:432] Initiating AddMaster RPC to add 127.8.18.252:41545
Aug 10 11:31:03 dist-test-slave-srgv krb5kdc[14545](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361459, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.8.18.193@KRBTEST.COM
I20260810 11:31:03.600521 15351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:03.560723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60896 (local address 127.8.18.193:44877)
0810 11:31:03.566771 (+ 6048us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:03.566784 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:03.566831 (+ 47us) server_negotiation.cc:407] Connection header received
0810 11:31:03.566943 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:03.566953 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:03.567419 (+ 466us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:03.570040 (+ 2621us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:31:03.570397 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:03.572743 (+ 2346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:03.575226 (+ 2483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:03.576084 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:03.576289 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:03.584515 (+ 8226us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:31:03.584550 (+ 35us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:31:03.584581 (+ 31us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:31:03.584669 (+ 88us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:31:03.592951 (+ 8282us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:31:03.594864 (+ 1913us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:31:03.594875 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:31:03.594879 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:31:03.595383 (+ 504us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:31:03.596221 (+ 838us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:31:03.596230 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:31:03.596236 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:31:03.596889 (+ 653us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:31:03.597572 (+ 683us) server_negotiation.cc:1035] Waiting for connection context
0810 11:31:03.599117 (+ 1545us) server_negotiation.cc:299] Negotiation successful
0810 11:31:03.599647 (+ 530us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5670,"thread_start_us":5493,"threads_started":1}
I20260810 11:31:03.603188 14783 generic_service.cc:196] Checking for leaks (request via RPC)
W20260810 11:31:03.631502 15283 tablet.cc:2369] T 00000000000000000000000000000000 P d1ea41131c2a4c6f984571036be55232: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 11:31:03.638319 15334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:03.581960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45106 (local address 127.8.18.252:41545)
0810 11:31:03.584395 (+ 2435us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:03.584401 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:03.586512 (+ 2111us) server_negotiation.cc:407] Connection header received
0810 11:31:03.587384 (+ 872us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:03.587404 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:03.587516 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:03.587611 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:31:03.598137 (+ 10526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:03.599859 (+ 1722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:03.605470 (+ 5611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:03.607226 (+ 1756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:03.607425 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:03.612121 (+ 4696us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:31:03.612129 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:31:03.612134 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:31:03.612190 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:31:03.621879 (+ 9689us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:31:03.632670 (+ 10791us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:31:03.632677 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:31:03.632681 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:31:03.632898 (+ 217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:31:03.633428 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:31:03.633433 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:31:03.633437 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:31:03.634158 (+ 721us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:31:03.634353 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0810 11:31:03.636879 (+ 2526us) server_negotiation.cc:299] Negotiation successful
0810 11:31:03.637715 (+ 836us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":537,"server-negotiator.queue_time_us":685}
I20260810 11:31:03.638868 15335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:03.571058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40462 (local address 127.8.18.254:36523)
0810 11:31:03.571279 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:03.571285 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:03.596341 (+ 25056us) server_negotiation.cc:407] Connection header received
0810 11:31:03.596837 (+ 496us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:03.596844 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:03.596934 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:03.597028 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:31:03.608695 (+ 11667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:03.610488 (+ 1793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:03.614070 (+ 3582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:03.615318 (+ 1248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:03.615467 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:03.621311 (+ 5844us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:31:03.621319 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:31:03.621324 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:31:03.621376 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:31:03.628247 (+ 6871us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:31:03.631531 (+ 3284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:31:03.631541 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:31:03.631546 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:31:03.631961 (+ 415us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:31:03.633597 (+ 1636us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:31:03.633605 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:31:03.633610 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:31:03.634383 (+ 773us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:31:03.634596 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0810 11:31:03.637639 (+ 3043us) server_negotiation.cc:299] Negotiation successful
0810 11:31:03.638006 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260810 11:31:03.639462 15358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:03.585535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:36523 (local address 127.0.0.1:40462)
0810 11:31:03.596146 (+ 10611us) negotiation.cc:107] Waiting for socket to connect
0810 11:31:03.596218 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0810 11:31:03.596567 (+ 349us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:31:03.597475 (+ 908us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:31:03.597482 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:31:03.602659 (+ 5177us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:31:03.608340 (+ 5681us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:03.608358 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:03.610814 (+ 2456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:03.610820 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:03.613806 (+ 2986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:03.613826 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:03.616132 (+ 2306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:03.616145 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:03.616520 (+ 375us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:03.617659 (+ 1139us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:31:03.617693 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:31:03.620929 (+ 3236us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:31:03.629197 (+ 8268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:31:03.629204 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:31:03.629208 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:31:03.630190 (+ 982us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:31:03.632169 (+ 1979us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:31:03.632174 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:31:03.632178 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:31:03.632481 (+ 303us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:31:03.635042 (+ 2561us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:31:03.635047 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:31:03.635401 (+ 354us) client_negotiation.cc:769] Sending connection context
0810 11:31:03.635631 (+ 230us) client_negotiation.cc:240] Negotiation successful
0810 11:31:03.636067 (+ 436us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10105,"spinlock_wait_cycles":37760,"thread_start_us":6855,"threads_started":1}
I20260810 11:31:03.641179 15265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:03.571811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47968 (local address 127.8.18.253:44341)
0810 11:31:03.572114 (+ 303us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:03.572120 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:03.573989 (+ 1869us) server_negotiation.cc:407] Connection header received
0810 11:31:03.574183 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:03.574190 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:03.574290 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:03.574431 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:31:03.588500 (+ 14069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:03.590373 (+ 1873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:03.599756 (+ 9383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:03.600850 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:03.600966 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:03.618931 (+ 17965us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:31:03.618942 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:31:03.618951 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:31:03.619033 (+ 82us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:31:03.626218 (+ 7185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:31:03.635762 (+ 9544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:31:03.635772 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:31:03.635777 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:31:03.636116 (+ 339us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:31:03.637735 (+ 1619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:31:03.637741 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:31:03.637744 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:31:03.638358 (+ 614us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:31:03.638535 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0810 11:31:03.640317 (+ 1782us) server_negotiation.cc:299] Negotiation successful
0810 11:31:03.640776 (+ 459us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162}
I20260810 11:31:03.642215 15357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:03.574174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.252:41545 (local address 127.0.0.1:45106)
0810 11:31:03.586338 (+ 12164us) negotiation.cc:107] Waiting for socket to connect
0810 11:31:03.586409 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0810 11:31:03.587057 (+ 648us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:31:03.587755 (+ 698us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:31:03.587759 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:31:03.595160 (+ 7401us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:31:03.597952 (+ 2792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:03.597975 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:03.601690 (+ 3715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:03.601696 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:03.605237 (+ 3541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:03.605265 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:03.607885 (+ 2620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:03.607908 (+ 23us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:03.608382 (+ 474us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:03.609652 (+ 1270us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:31:03.609678 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:31:03.611845 (+ 2167us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:31:03.630385 (+ 18540us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:31:03.630390 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:31:03.630394 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:31:03.631305 (+ 911us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:31:03.633046 (+ 1741us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:31:03.633051 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:31:03.633054 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:31:03.633271 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:31:03.634436 (+ 1165us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:31:03.634441 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:31:03.634690 (+ 249us) client_negotiation.cc:769] Sending connection context
0810 11:31:03.636667 (+ 1977us) client_negotiation.cc:240] Negotiation successful
0810 11:31:03.638721 (+ 2054us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11511,"spinlock_wait_cycles":28544,"thread_start_us":10734,"threads_started":1}
I20260810 11:31:03.647605 15356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:03.571458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.253:44341 (local address 127.0.0.1:47968)
0810 11:31:03.573794 (+ 2336us) negotiation.cc:107] Waiting for socket to connect
0810 11:31:03.573858 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0810 11:31:03.574026 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:31:03.574508 (+ 482us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:31:03.574517 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:31:03.586048 (+ 11531us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:31:03.588192 (+ 2144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:03.588222 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:03.590963 (+ 2741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:03.590970 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:03.599535 (+ 8565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:03.599558 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:03.601625 (+ 2067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:03.601639 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:03.602016 (+ 377us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:03.603444 (+ 1428us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:31:03.603471 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:31:03.618483 (+ 15012us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:31:03.634695 (+ 16212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:31:03.634702 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:31:03.634707 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:31:03.635514 (+ 807us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:31:03.636353 (+ 839us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:31:03.636360 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:31:03.636365 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:31:03.637556 (+ 1191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:31:03.638664 (+ 1108us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:31:03.638675 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:31:03.639171 (+ 496us) client_negotiation.cc:769] Sending connection context
0810 11:31:03.641984 (+ 2813us) client_negotiation.cc:240] Negotiation successful
0810 11:31:03.643144 (+ 1160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2090,"thread_start_us":1638,"threads_started":1}
I20260810 11:31:03.660316 15338 raft_consensus.cc:482] T 00000000000000000000000000000000 P d1ea41131c2a4c6f984571036be55232 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 11:31:03.660954 15338 raft_consensus.cc:504] T 00000000000000000000000000000000 P d1ea41131c2a4c6f984571036be55232 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd8ee5823e444d28a6cb100898b494c5" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 36523 } } peers { permanent_uuid: "8fb82cee99b449c5a33adb02cba79748" member_type: VOTER last_known_addr { host: "127.8.18.253" port: 44341 } } peers { permanent_uuid: "d1ea41131c2a4c6f984571036be55232" member_type: VOTER last_known_addr { host: "127.8.18.252" port: 41545 } }
I20260810 11:31:03.664471 15220 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d1ea41131c2a4c6f984571036be55232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fb82cee99b449c5a33adb02cba79748" is_pre_election: true
I20260810 11:31:03.664693 14601 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d1ea41131c2a4c6f984571036be55232" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd8ee5823e444d28a6cb100898b494c5" is_pre_election: true
I20260810 11:31:03.666492 15338 leader_election.cc:288] T 00000000000000000000000000000000 P d1ea41131c2a4c6f984571036be55232 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd8ee5823e444d28a6cb100898b494c5 (127.8.18.254:36523), 8fb82cee99b449c5a33adb02cba79748 (127.8.18.253:44341)
I20260810 11:31:03.666672 15281 leader_election.cc:302] T 00000000000000000000000000000000 P d1ea41131c2a4c6f984571036be55232 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1ea41131c2a4c6f984571036be55232; no voters: 8fb82cee99b449c5a33adb02cba79748, cd8ee5823e444d28a6cb100898b494c5
I20260810 11:31:03.667418 15338 raft_consensus.cc:2726] T 00000000000000000000000000000000 P d1ea41131c2a4c6f984571036be55232 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
Aug 10 11:31:03 dist-test-slave-srgv krb5kdc[14545](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361459, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.18.254@KRBTEST.COM for oryx/127.8.18.252@KRBTEST.COM
I20260810 11:31:03.684722 15334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:03.651246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45114 (local address 127.8.18.252:41545)
0810 11:31:03.651480 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:03.651488 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:03.651674 (+ 186us) server_negotiation.cc:407] Connection header received
0810 11:31:03.651945 (+ 271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:03.651951 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:03.652138 (+ 187us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:03.652268 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:31:03.654119 (+ 1851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:03.655833 (+ 1714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:03.658799 (+ 2966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:03.660421 (+ 1622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:03.660648 (+ 227us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:03.671826 (+ 11178us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:31:03.671835 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:31:03.671841 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:31:03.671891 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:31:03.679145 (+ 7254us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:31:03.680877 (+ 1732us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:31:03.680885 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:31:03.680889 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:31:03.681159 (+ 270us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:31:03.682005 (+ 846us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:31:03.682011 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:31:03.682015 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:31:03.682579 (+ 564us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:31:03.682920 (+ 341us) server_negotiation.cc:1035] Waiting for connection context
0810 11:31:03.683685 (+ 765us) server_negotiation.cc:299] Negotiation successful
0810 11:31:03.684102 (+ 417us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260810 11:31:03.684773 15264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:03.651103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.252:41545 (local address 127.0.0.1:45114)
0810 11:31:03.651568 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0810 11:31:03.651590 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 11:31:03.651751 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:31:03.652337 (+ 586us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:31:03.652343 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:31:03.653314 (+ 971us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:31:03.653816 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:03.653834 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:03.656096 (+ 2262us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:03.656103 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:03.658406 (+ 2303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:03.658437 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:03.660943 (+ 2506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:03.660951 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:03.661326 (+ 375us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:03.662826 (+ 1500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:31:03.662857 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:31:03.671324 (+ 8467us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:31:03.679472 (+ 8148us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:31:03.679481 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:31:03.679486 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:31:03.680641 (+ 1155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:31:03.681358 (+ 717us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:31:03.681365 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:31:03.681368 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:31:03.681771 (+ 403us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:31:03.682900 (+ 1129us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:31:03.682907 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:31:03.683254 (+ 347us) client_negotiation.cc:769] Sending connection context
0810 11:31:03.683517 (+ 263us) client_negotiation.cc:240] Negotiation successful
0810 11:31:03.684181 (+ 664us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277}
I20260810 11:31:03.688027 14591 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.8.18.252:41545: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "d1ea41131c2a4c6f984571036be55232"
member_type: NON_VOTER
last_known_addr {
host: "127.8.18.252"
port: 41545
}
attrs {
promote: true
}
}
dest_uuid: "cd8ee5823e444d28a6cb100898b494c5"
cas_config_opid_index: 5
I20260810 11:31:03.689993 14591 consensus_queue.cc:237] T 00000000000000000000000000000000 P cd8ee5823e444d28a6cb100898b494c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "cd8ee5823e444d28a6cb100898b494c5" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 36523 } } peers { permanent_uuid: "8fb82cee99b449c5a33adb02cba79748" member_type: NON_VOTER last_known_addr { host: "127.8.18.253" port: 44341 } attrs { promote: true } } peers { permanent_uuid: "d1ea41131c2a4c6f984571036be55232" member_type: NON_VOTER last_known_addr { host: "127.8.18.252" port: 41545 } attrs { promote: true } }
I20260810 11:31:03.694383 15260 raft_consensus.cc:2932] T 00000000000000000000000000000000 P cd8ee5823e444d28a6cb100898b494c5 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER d1ea41131c2a4c6f984571036be55232 (127.8.18.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "cd8ee5823e444d28a6cb100898b494c5" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 36523 } } peers { permanent_uuid: "8fb82cee99b449c5a33adb02cba79748" member_type: NON_VOTER last_known_addr { host: "127.8.18.253" port: 44341 } attrs { promote: true } } peers { permanent_uuid: "d1ea41131c2a4c6f984571036be55232" member_type: NON_VOTER last_known_addr { host: "127.8.18.252" port: 41545 } attrs { promote: true } } }
I20260810 11:31:03.695253 15305 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d1ea41131c2a4c6f984571036be55232 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:31:03.702559 15305 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d1ea41131c2a4c6f984571036be55232 [term 1 FOLLOWER]: Refusing update from remote peer cd8ee5823e444d28a6cb100898b494c5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260810 11:31:03.714998 15364 consensus_queue.cc:1035] T 00000000000000000000000000000000 P cd8ee5823e444d28a6cb100898b494c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1ea41131c2a4c6f984571036be55232" member_type: NON_VOTER last_known_addr { host: "127.8.18.252" port: 41545 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.001s
I20260810 11:31:03.716990 15260 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.8.18.252:41545
I20260810 11:31:03.718974 15220 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8fb82cee99b449c5a33adb02cba79748 [term 1 LEARNER]: Refusing update from remote peer cd8ee5823e444d28a6cb100898b494c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260810 11:31:03.719172 15178 master.cc:537] Master@127.8.18.252:41545 shutting down...
I20260810 11:31:03.722183 15361 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd8ee5823e444d28a6cb100898b494c5 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "cd8ee5823e444d28a6cb100898b494c5" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "cd8ee5823e444d28a6cb100898b494c5" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 36523 } } peers { permanent_uuid: "8fb82cee99b449c5a33adb02cba79748" member_type: NON_VOTER last_known_addr { host: "127.8.18.253" port: 44341 } attrs { promote: true } } peers { permanent_uuid: "d1ea41131c2a4c6f984571036be55232" member_type: NON_VOTER last_known_addr { host: "127.8.18.252" port: 41545 } attrs { promote: true } } }
I20260810 11:31:03.723451 15361 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd8ee5823e444d28a6cb100898b494c5 [sys.catalog]: This master's current role is: LEADER
I20260810 11:31:03.725732 15363 consensus_queue.cc:1035] T 00000000000000000000000000000000 P cd8ee5823e444d28a6cb100898b494c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fb82cee99b449c5a33adb02cba79748" member_type: NON_VOTER last_known_addr { host: "127.8.18.253" port: 44341 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.004s
I20260810 11:31:03.736135 15220 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 8fb82cee99b449c5a33adb02cba79748 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER d1ea41131c2a4c6f984571036be55232 (127.8.18.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "cd8ee5823e444d28a6cb100898b494c5" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 36523 } } peers { permanent_uuid: "8fb82cee99b449c5a33adb02cba79748" member_type: NON_VOTER last_known_addr { host: "127.8.18.253" port: 44341 } attrs { promote: true } } peers { permanent_uuid: "d1ea41131c2a4c6f984571036be55232" member_type: NON_VOTER last_known_addr { host: "127.8.18.252" port: 41545 } attrs { promote: true } } }
I20260810 11:31:03.743829 15248 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8fb82cee99b449c5a33adb02cba79748 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "cd8ee5823e444d28a6cb100898b494c5" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "cd8ee5823e444d28a6cb100898b494c5" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 36523 } } peers { permanent_uuid: "8fb82cee99b449c5a33adb02cba79748" member_type: NON_VOTER last_known_addr { host: "127.8.18.253" port: 44341 } attrs { promote: true } } peers { permanent_uuid: "d1ea41131c2a4c6f984571036be55232" member_type: NON_VOTER last_known_addr { host: "127.8.18.252" port: 41545 } attrs { promote: true } } }
I20260810 11:31:03.744421 15248 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8fb82cee99b449c5a33adb02cba79748 [sys.catalog]: This master's current role is: LEARNER
I20260810 11:31:03.745970 15364 raft_consensus.cc:1050] T 00000000000000000000000000000000 P cd8ee5823e444d28a6cb100898b494c5: attempting to promote NON_VOTER 8fb82cee99b449c5a33adb02cba79748 to VOTER
I20260810 11:31:03.747754 15364 consensus_queue.cc:237] T 00000000000000000000000000000000 P cd8ee5823e444d28a6cb100898b494c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "cd8ee5823e444d28a6cb100898b494c5" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 36523 } } peers { permanent_uuid: "8fb82cee99b449c5a33adb02cba79748" member_type: VOTER last_known_addr { host: "127.8.18.253" port: 44341 } attrs { promote: false } } peers { permanent_uuid: "d1ea41131c2a4c6f984571036be55232" member_type: NON_VOTER last_known_addr { host: "127.8.18.252" port: 41545 } attrs { promote: true } }
I20260810 11:31:03.751657 15220 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8fb82cee99b449c5a33adb02cba79748 [term 1 LEARNER]: Refusing update from remote peer cd8ee5823e444d28a6cb100898b494c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260810 11:31:03.752719 15361 consensus_queue.cc:1035] T 00000000000000000000000000000000 P cd8ee5823e444d28a6cb100898b494c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fb82cee99b449c5a33adb02cba79748" member_type: VOTER last_known_addr { host: "127.8.18.253" port: 44341 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260810 11:31:03.760277 15178 raft_consensus.cc:2229] T 00000000000000000000000000000000 P d1ea41131c2a4c6f984571036be55232 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 11:31:03.760952 15178 raft_consensus.cc:2258] T 00000000000000000000000000000000 P d1ea41131c2a4c6f984571036be55232 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:31:03.761445 15178 tablet_replica.cc:330] T 00000000000000000000000000000000 P d1ea41131c2a4c6f984571036be55232: stopping tablet replica
I20260810 11:31:03.770236 15264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:03.769540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.252:41545 (local address 127.0.0.1:45118)
0810 11:31:03.769731 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0810 11:31:03.769873 (+ 142us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.252:41545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260810 11:31:03.770689 14576 consensus_peers.cc:487] T 00000000000000000000000000000000 P cd8ee5823e444d28a6cb100898b494c5 -> Peer d1ea41131c2a4c6f984571036be55232 (127.8.18.252:41545): Couldn't send request to peer d1ea41131c2a4c6f984571036be55232. Status: Network error: Client connection negotiation failed: client connection to 127.8.18.252:41545: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260810 11:31:03.772233 15363 raft_consensus.cc:2932] T 00000000000000000000000000000000 P cd8ee5823e444d28a6cb100898b494c5 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 8fb82cee99b449c5a33adb02cba79748 (127.8.18.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "cd8ee5823e444d28a6cb100898b494c5" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 36523 } } peers { permanent_uuid: "8fb82cee99b449c5a33adb02cba79748" member_type: VOTER last_known_addr { host: "127.8.18.253" port: 44341 } attrs { promote: false } } peers { permanent_uuid: "d1ea41131c2a4c6f984571036be55232" member_type: NON_VOTER last_known_addr { host: "127.8.18.252" port: 41545 } attrs { promote: true } } }
I20260810 11:31:03.773627 15220 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 8fb82cee99b449c5a33adb02cba79748 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 8fb82cee99b449c5a33adb02cba79748 (127.8.18.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "cd8ee5823e444d28a6cb100898b494c5" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 36523 } } peers { permanent_uuid: "8fb82cee99b449c5a33adb02cba79748" member_type: VOTER last_known_addr { host: "127.8.18.253" port: 44341 } attrs { promote: false } } peers { permanent_uuid: "d1ea41131c2a4c6f984571036be55232" member_type: NON_VOTER last_known_addr { host: "127.8.18.252" port: 41545 } attrs { promote: true } } }
I20260810 11:31:03.778666 15364 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd8ee5823e444d28a6cb100898b494c5 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "cd8ee5823e444d28a6cb100898b494c5" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "cd8ee5823e444d28a6cb100898b494c5" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 36523 } } peers { permanent_uuid: "8fb82cee99b449c5a33adb02cba79748" member_type: VOTER last_known_addr { host: "127.8.18.253" port: 44341 } attrs { promote: false } } peers { permanent_uuid: "d1ea41131c2a4c6f984571036be55232" member_type: NON_VOTER last_known_addr { host: "127.8.18.252" port: 41545 } attrs { promote: true } } }
I20260810 11:31:03.779502 15364 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd8ee5823e444d28a6cb100898b494c5 [sys.catalog]: This master's current role is: LEADER
I20260810 11:31:03.780107 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 14655
I20260810 11:31:03.780350 15248 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8fb82cee99b449c5a33adb02cba79748 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "cd8ee5823e444d28a6cb100898b494c5" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "cd8ee5823e444d28a6cb100898b494c5" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 36523 } } peers { permanent_uuid: "8fb82cee99b449c5a33adb02cba79748" member_type: VOTER last_known_addr { host: "127.8.18.253" port: 44341 } attrs { promote: false } } peers { permanent_uuid: "d1ea41131c2a4c6f984571036be55232" member_type: NON_VOTER last_known_addr { host: "127.8.18.252" port: 41545 } attrs { promote: true } } }
I20260810 11:31:03.780745 15248 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8fb82cee99b449c5a33adb02cba79748 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 11:31:03.781944 15361 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd8ee5823e444d28a6cb100898b494c5 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "cd8ee5823e444d28a6cb100898b494c5" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "cd8ee5823e444d28a6cb100898b494c5" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 36523 } } peers { permanent_uuid: "8fb82cee99b449c5a33adb02cba79748" member_type: VOTER last_known_addr { host: "127.8.18.253" port: 44341 } attrs { promote: false } } peers { permanent_uuid: "d1ea41131c2a4c6f984571036be55232" member_type: NON_VOTER last_known_addr { host: "127.8.18.252" port: 41545 } attrs { promote: true } } }
I20260810 11:31:03.782464 15361 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd8ee5823e444d28a6cb100898b494c5 [sys.catalog]: This master's current role is: LEADER
I20260810 11:31:03.790714 15178 master.cc:552] Master@127.8.18.252:41545 shutdown complete.
I20260810 11:31:03.790967 15178 master_runner.cc:305] Clearing existing system tablet
I20260810 11:31:03.791745 15178 fs_manager.cc:359] Metadata directory not provided
I20260810 11:31:03.791980 15178 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:31:03.802166 15178 fs_manager.cc:709] Time spent opening directory manager: real 0.009s user 0.011s sys 0.001s
I20260810 11:31:03.802773 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.18.193:44877
--local_ip_for_outbound_sockets=127.8.18.193
--tserver_master_addrs=127.8.18.254:36523
--webserver_port=36315
--webserver_interface=127.8.18.193
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/krb5kdc/oryx_127.8.18.193.keytab
--principal=oryx/127.8.18.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.18.212:34803
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.8.18.254:36523,127.8.18.253:44341
--tserver_master_addrs=127.8.18.254:36523,127.8.18.253:44341,127.8.18.252:41545 with env {KRB5CCNAME=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 11:31:03.805331 15369 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:31:03.805989 15178 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:31:03.806483 15178 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260810 11:31:03.806777 15178 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/master-2/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/master-2/wal
uuid: "d1ea41131c2a4c6f984571036be55232"
format_stamp: "Formatted at 2026-08-10 11:31:03 on dist-test-slave-srgv"
server_key: "7c97d9bd2cf22ac2d531541b5cf22a8d"
server_key_iv: "767195ad4a0e0f0255eda1652ac571c1"
server_key_version: "encryptionkey@0"
I20260810 11:31:03.814827 15178 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P d1ea41131c2a4c6f984571036be55232: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:31:03.827577 15178 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P d1ea41131c2a4c6f984571036be55232: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260810 11:31:03.827953 15178 log.cc:1196] T 00000000000000000000000000000000 P d1ea41131c2a4c6f984571036be55232: Deleting WAL directory at /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260810 11:31:03.829095 15178 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P d1ea41131c2a4c6f984571036be55232: Deleting consensus metadata
I20260810 11:31:03.829466 15178 master_runner.cc:315] Copying system tablet from 127.8.18.254:36523
I20260810 11:31:03.835501 15178 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P d1ea41131c2a4c6f984571036be55232: tablet copy: Beginning tablet copy session from remote peer at address 127.8.18.254:36523
I20260810 11:31:03.868160 15376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:03.836947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:36523 (local address 127.0.0.1:40474)
0810 11:31:03.838341 (+ 1394us) negotiation.cc:107] Waiting for socket to connect
0810 11:31:03.838415 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0810 11:31:03.838817 (+ 402us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:31:03.839611 (+ 794us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:31:03.839616 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:31:03.840616 (+ 1000us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:31:03.841368 (+ 752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:03.841393 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:03.844715 (+ 3322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:03.844725 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:03.847407 (+ 2682us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:03.847456 (+ 49us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:03.849790 (+ 2334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:03.849796 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:03.850143 (+ 347us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:03.851458 (+ 1315us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:31:03.851487 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:31:03.853782 (+ 2295us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:31:03.862477 (+ 8695us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:31:03.862484 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:31:03.862488 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:31:03.863600 (+ 1112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:31:03.864550 (+ 950us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:31:03.864556 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:31:03.864564 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:31:03.865037 (+ 473us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:31:03.866109 (+ 1072us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:31:03.866128 (+ 19us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:31:03.866493 (+ 365us) client_negotiation.cc:769] Sending connection context
0810 11:31:03.866774 (+ 281us) client_negotiation.cc:240] Negotiation successful
0810 11:31:03.867300 (+ 526us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1165,"thread_start_us":860,"threads_started":1}
I20260810 11:31:03.868458 15335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:03.837539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40474 (local address 127.8.18.254:36523)
0810 11:31:03.837923 (+ 384us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:03.837930 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:03.838541 (+ 611us) server_negotiation.cc:407] Connection header received
0810 11:31:03.839072 (+ 531us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:03.839080 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:03.839260 (+ 180us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:03.839350 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:31:03.841700 (+ 2350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:03.843981 (+ 2281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:03.847775 (+ 3794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:03.849154 (+ 1379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:03.849455 (+ 301us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:03.854258 (+ 4803us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:31:03.854285 (+ 27us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:31:03.854292 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:31:03.854356 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:31:03.862142 (+ 7786us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:31:03.863885 (+ 1743us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:31:03.863904 (+ 19us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:31:03.863909 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:31:03.864337 (+ 428us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:31:03.865275 (+ 938us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:31:03.865282 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:31:03.865285 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:31:03.865858 (+ 573us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:31:03.866222 (+ 364us) server_negotiation.cc:1035] Waiting for connection context
0810 11:31:03.867170 (+ 948us) server_negotiation.cc:299] Negotiation successful
0810 11:31:03.867620 (+ 450us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":235}
I20260810 11:31:03.869716 14612 tablet_copy_service.cc:140] P cd8ee5823e444d28a6cb100898b494c5: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer d1ea41131c2a4c6f984571036be55232 ({username='oryx', principal='oryx/127.8.18.252@KRBTEST.COM'} at 127.0.0.1:40474)
I20260810 11:31:03.870031 14612 tablet_copy_service.cc:161] P cd8ee5823e444d28a6cb100898b494c5: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer d1ea41131c2a4c6f984571036be55232 at {username='oryx', principal='oryx/127.8.18.252@KRBTEST.COM'} at 127.0.0.1:40474: session id = d1ea41131c2a4c6f984571036be55232-00000000000000000000000000000000
I20260810 11:31:03.877053 14612 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P cd8ee5823e444d28a6cb100898b494c5: Tablet Copy: opened 0 blocks and 1 log segments
I20260810 11:31:03.880327 15178 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260810 11:31:03.894411 15178 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P d1ea41131c2a4c6f984571036be55232: tablet copy: Starting download of 0 data blocks...
I20260810 11:31:03.895067 15178 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P d1ea41131c2a4c6f984571036be55232: tablet copy: Starting download of 1 WAL segments...
I20260810 11:31:03.901381 15178 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P d1ea41131c2a4c6f984571036be55232: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260810 11:31:03.909971 14612 tablet_copy_service.cc:342] P cd8ee5823e444d28a6cb100898b494c5: Request end of tablet copy session d1ea41131c2a4c6f984571036be55232-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.8.18.252@KRBTEST.COM'} at 127.0.0.1:40474
I20260810 11:31:03.910319 14612 tablet_copy_service.cc:433] P cd8ee5823e444d28a6cb100898b494c5: ending tablet copy session d1ea41131c2a4c6f984571036be55232-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer d1ea41131c2a4c6f984571036be55232
I20260810 11:31:03.926649 15178 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:31:03.929998 15378 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:31:03.938189 15379 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:31:03.942179 15178 server_base.cc:1034] running on GCE node
W20260810 11:31:03.943820 15381 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:31:03.950120 15178 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:31:03.970283 15178 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:31:03.971701 15178 hybrid_clock.cc:648] HybridClock initialized: now 1786361463971657 us; error 44 us; skew 500 ppm
I20260810 11:31:03.972206 15178 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:31:03 dist-test-slave-srgv krb5kdc[14545](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361463, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.18.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:31:03.975207 15178 init.cc:377] Logged in from keytab as oryx/127.8.18.252@KRBTEST.COM (short username oryx)
I20260810 11:31:03.982810 15178 webserver.cc:460] Webserver started at http://127.8.18.252:37355/ using document root <none> and password file <none>
I20260810 11:31:03.983409 15178 fs_manager.cc:359] Metadata directory not provided
I20260810 11:31:03.983632 15178 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:31:03.987576 15178 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260810 11:31:03.990343 15388 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:31:03.991108 15178 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.001s
I20260810 11:31:03.991385 15178 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/master-2/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/master-2/wal
uuid: "d1ea41131c2a4c6f984571036be55232"
format_stamp: "Formatted at 2026-08-10 11:31:03 on dist-test-slave-srgv"
server_key: "7c97d9bd2cf22ac2d531541b5cf22a8d"
server_key_iv: "767195ad4a0e0f0255eda1652ac571c1"
server_key_version: "encryptionkey@0"
I20260810 11:31:03.991818 15178 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:31:04.029011 15178 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:31:04.034636 15178 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:31:04.108852 15178 rpc_server.cc:307] RPC server started. Bound to: 127.8.18.252:41545
I20260810 11:31:04.108939 15440 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.18.252:41545 every 8 connection(s)
I20260810 11:31:04.110170 15178 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/master-2/data/info.pb
I20260810 11:31:04.116089 15441 sys_catalog.cc:263] Verifying existing consensus state
I20260810 11:31:04.120699 15441 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d1ea41131c2a4c6f984571036be55232: Bootstrap starting.
I20260810 11:31:04.146466 15441 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d1ea41131c2a4c6f984571036be55232: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 11:31:04.147109 15441 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d1ea41131c2a4c6f984571036be55232: Bootstrap complete.
I20260810 11:31:04.148321 15441 raft_consensus.cc:348] T 00000000000000000000000000000000 P d1ea41131c2a4c6f984571036be55232 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "cd8ee5823e444d28a6cb100898b494c5" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 36523 } } peers { permanent_uuid: "8fb82cee99b449c5a33adb02cba79748" member_type: VOTER last_known_addr { host: "127.8.18.253" port: 44341 } attrs { promote: false } } peers { permanent_uuid: "d1ea41131c2a4c6f984571036be55232" member_type: NON_VOTER last_known_addr { host: "127.8.18.252" port: 41545 } attrs { promote: true } }
I20260810 11:31:04.148712 15441 raft_consensus.cc:729] T 00000000000000000000000000000000 P d1ea41131c2a4c6f984571036be55232 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: d1ea41131c2a4c6f984571036be55232, State: Initialized, Role: LEARNER
I20260810 11:31:04.149046 15441 consensus_queue.cc:260] T 00000000000000000000000000000000 P d1ea41131c2a4c6f984571036be55232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "cd8ee5823e444d28a6cb100898b494c5" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 36523 } } peers { permanent_uuid: "8fb82cee99b449c5a33adb02cba79748" member_type: VOTER last_known_addr { host: "127.8.18.253" port: 44341 } attrs { promote: false } } peers { permanent_uuid: "d1ea41131c2a4c6f984571036be55232" member_type: NON_VOTER last_known_addr { host: "127.8.18.252" port: 41545 } attrs { promote: true } }
I20260810 11:31:04.150141 15444 sys_catalog.cc:455] T 00000000000000000000000000000000 P d1ea41131c2a4c6f984571036be55232 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "cd8ee5823e444d28a6cb100898b494c5" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 36523 } } peers { permanent_uuid: "8fb82cee99b449c5a33adb02cba79748" member_type: VOTER last_known_addr { host: "127.8.18.253" port: 44341 } attrs { promote: false } } peers { permanent_uuid: "d1ea41131c2a4c6f984571036be55232" member_type: NON_VOTER last_known_addr { host: "127.8.18.252" port: 41545 } attrs { promote: true } } }
I20260810 11:31:04.150465 15441 sys_catalog.cc:564] T 00000000000000000000000000000000 P d1ea41131c2a4c6f984571036be55232 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 11:31:04.150588 15444 sys_catalog.cc:458] T 00000000000000000000000000000000 P d1ea41131c2a4c6f984571036be55232 [sys.catalog]: This master's current role is: LEARNER
I20260810 11:31:04.159580 15455 catalog_manager.cc:1259] Loaded cluster ID: 63349ed0186a4282894be80560e2d468
I20260810 11:31:04.159869 15455 catalog_manager.cc:1552] T 00000000000000000000000000000000 P d1ea41131c2a4c6f984571036be55232: loading cluster ID for follower catalog manager: success
I20260810 11:31:04.163997 15455 catalog_manager.cc:1574] T 00000000000000000000000000000000 P d1ea41131c2a4c6f984571036be55232: acquiring CA information for follower catalog manager: success
I20260810 11:31:04.167147 15455 catalog_manager.cc:1602] T 00000000000000000000000000000000 P d1ea41131c2a4c6f984571036be55232: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260810 11:31:04.371126 15370 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:31:04.371800 15370 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:31:04.372161 15370 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260810 11:31:04.376784 15460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:04.365760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.252:41545 (local address 127.0.0.1:45120)
0810 11:31:04.366998 (+ 1238us) negotiation.cc:107] Waiting for socket to connect
0810 11:31:04.367189 (+ 191us) client_negotiation.cc:174] Beginning negotiation
0810 11:31:04.367392 (+ 203us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:31:04.368052 (+ 660us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:31:04.368058 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:31:04.368167 (+ 109us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:31:04.368788 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:04.368805 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:04.370587 (+ 1782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:04.370593 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:04.373504 (+ 2911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:04.373534 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:04.375448 (+ 1914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:04.375457 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:04.375898 (+ 441us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:04.375952 (+ 54us) client_negotiation.cc:769] Sending connection context
0810 11:31:04.376147 (+ 195us) client_negotiation.cc:240] Negotiation successful
0810 11:31:04.376345 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1015,"thread_start_us":823,"threads_started":1}
I20260810 11:31:04.377465 15461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:04.366018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45120 (local address 127.8.18.252:41545)
0810 11:31:04.367662 (+ 1644us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:04.367667 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:04.367687 (+ 20us) server_negotiation.cc:407] Connection header received
0810 11:31:04.367755 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:04.367759 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:04.367836 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:04.367980 (+ 144us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:31:04.369015 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:04.370389 (+ 1374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:04.373831 (+ 3442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:04.375052 (+ 1221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:04.375246 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:04.375477 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0810 11:31:04.376280 (+ 803us) server_negotiation.cc:299] Negotiation successful
0810 11:31:04.376480 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1461,"thread_start_us":1290,"threads_started":1}
I20260810 11:31:04.379247 15416 raft_consensus.cc:1203] T 00000000000000000000000000000000 P d1ea41131c2a4c6f984571036be55232 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260810 11:31:04.380079 15444 sys_catalog.cc:455] T 00000000000000000000000000000000 P d1ea41131c2a4c6f984571036be55232 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cd8ee5823e444d28a6cb100898b494c5. Latest consensus state: current_term: 1 leader_uuid: "cd8ee5823e444d28a6cb100898b494c5" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "cd8ee5823e444d28a6cb100898b494c5" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 36523 } } peers { permanent_uuid: "8fb82cee99b449c5a33adb02cba79748" member_type: VOTER last_known_addr { host: "127.8.18.253" port: 44341 } attrs { promote: false } } peers { permanent_uuid: "d1ea41131c2a4c6f984571036be55232" member_type: NON_VOTER last_known_addr { host: "127.8.18.252" port: 41545 } attrs { promote: true } } }
I20260810 11:31:04.380460 15444 sys_catalog.cc:458] T 00000000000000000000000000000000 P d1ea41131c2a4c6f984571036be55232 [sys.catalog]: This master's current role is: LEARNER
W20260810 11:31:04.385856 15370 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:31:04.386188 15370 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:31:04.386498 15370 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.18.193
I20260810 11:31:04.402740 15370 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.18.212:34803
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/krb5kdc/oryx_127.8.18.193.keytab
--principal=oryx/127.8.18.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.18.193:44877
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.18.193
--webserver_port=36315
--tserver_master_addrs=127.8.18.254:36523,127.8.18.253:44341,127.8.18.252:41545
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.18.193
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:31:04.404834 15370 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:31:04.406937 15370 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:31:04.430464 15463 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:31:04.431504 15466 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:31:04.432271 15464 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:31:04.433995 15370 server_base.cc:1034] running on GCE node
I20260810 11:31:04.435014 15370 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:31:04.436537 15370 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:31:04.437944 15370 hybrid_clock.cc:648] HybridClock initialized: now 1786361464437793 us; error 64 us; skew 500 ppm
I20260810 11:31:04.438505 15370 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:31:04 dist-test-slave-srgv krb5kdc[14545](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361464, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.18.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:31:04.444643 15370 init.cc:377] Logged in from keytab as oryx/127.8.18.193@KRBTEST.COM (short username oryx)
I20260810 11:31:04.447440 15370 webserver.cc:460] Webserver started at http://127.8.18.193:36315/ using document root <none> and password file <none>
I20260810 11:31:04.448546 15370 fs_manager.cc:359] Metadata directory not provided
I20260810 11:31:04.448782 15370 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:31:04.457881 15370 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.002s sys 0.005s
I20260810 11:31:04.462744 15473 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:31:04.464809 15370 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.002s
I20260810 11:31:04.465193 15370 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/ts-0/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/ts-0/wal
uuid: "b087d7b7ed0f44b585cbb87fa59cda8b"
format_stamp: "Formatted at 2026-08-10 11:31:00 on dist-test-slave-srgv"
server_key: "2d47d0d5f71a93dd8251d884ae2ed8b8"
server_key_iv: "81d40e980f58f56d6dc4e47e07cf3f82"
server_key_version: "encryptionkey@0"
I20260810 11:31:04.467293 15370 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:31:04.504787 15370 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:31:04.512761 15370 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:31:04.513300 15370 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:31:04.515326 15370 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:31:04.517899 15370 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:31:04.518115 15370 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:31:04.518245 15370 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:31:04.518375 15370 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:31:04.614468 15370 rpc_server.cc:307] RPC server started. Bound to: 127.8.18.193:44877
I20260810 11:31:04.614722 15590 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.18.193:44877 every 8 connection(s)
I20260810 11:31:04.616596 15370 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/ts-0/data/info.pb
I20260810 11:31:04.625124 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 15370
I20260810 11:31:04.625926 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 14794
Aug 10 11:31:04 dist-test-slave-srgv krb5kdc[14545](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361459, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.8.18.194@KRBTEST.COM
Aug 10 11:31:04 dist-test-slave-srgv krb5kdc[14545](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361464, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.18.193@KRBTEST.COM for oryx/127.8.18.252@KRBTEST.COM
Aug 10 11:31:04 dist-test-slave-srgv krb5kdc[14545](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361464, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.18.193@KRBTEST.COM for oryx/127.8.18.253@KRBTEST.COM
Aug 10 11:31:04 dist-test-slave-srgv krb5kdc[14545](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361464, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.18.193@KRBTEST.COM for oryx/127.8.18.254@KRBTEST.COM
I20260810 11:31:04.667680 15600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:04.628372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55076 (local address 127.8.18.194:40169)
0810 11:31:04.629548 (+ 1176us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:04.629559 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:04.634489 (+ 4930us) server_negotiation.cc:407] Connection header received
0810 11:31:04.634728 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:04.634741 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:04.635239 (+ 498us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:04.635459 (+ 220us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:31:04.639905 (+ 4446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:04.642266 (+ 2361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:04.645583 (+ 3317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:04.646895 (+ 1312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:04.647160 (+ 265us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:04.656012 (+ 8852us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:31:04.656036 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:31:04.656052 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:31:04.656097 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:31:04.662742 (+ 6645us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:31:04.664279 (+ 1537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:31:04.664287 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:31:04.664290 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:31:04.664544 (+ 254us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:31:04.665097 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:31:04.665100 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:31:04.665102 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:31:04.665655 (+ 553us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:31:04.666055 (+ 400us) server_negotiation.cc:1035] Waiting for connection context
0810 11:31:04.666462 (+ 407us) server_negotiation.cc:299] Negotiation successful
0810 11:31:04.666849 (+ 387us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":921,"thread_start_us":749,"threads_started":1}
I20260810 11:31:04.669864 14921 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:31:04.673153 15461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:04.624408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.193:35399 (local address 127.8.18.252:41545)
0810 11:31:04.624750 (+ 342us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:04.624759 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:04.639482 (+ 14723us) server_negotiation.cc:407] Connection header received
0810 11:31:04.642362 (+ 2880us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:04.642370 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:04.642474 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:04.642556 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:31:04.648086 (+ 5530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:04.649659 (+ 1573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:04.654871 (+ 5212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:04.656161 (+ 1290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:04.656288 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:04.662980 (+ 6692us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:31:04.662988 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:31:04.662993 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:31:04.663046 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:31:04.667591 (+ 4545us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:31:04.668944 (+ 1353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:31:04.668951 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:31:04.668956 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:31:04.669333 (+ 377us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:31:04.670309 (+ 976us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:31:04.670314 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:31:04.670317 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:31:04.670798 (+ 481us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:31:04.670961 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0810 11:31:04.672249 (+ 1288us) server_negotiation.cc:299] Negotiation successful
0810 11:31:04.672585 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116}
I20260810 11:31:04.675011 15601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:04.639871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.193:51279 (local address 127.8.18.253:44341)
0810 11:31:04.640992 (+ 1121us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:04.641000 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:04.642155 (+ 1155us) server_negotiation.cc:407] Connection header received
0810 11:31:04.642449 (+ 294us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:04.642454 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:04.642603 (+ 149us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:04.642728 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:31:04.648102 (+ 5374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:04.649235 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:04.656986 (+ 7751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:04.658132 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:04.658259 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:04.663468 (+ 5209us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:31:04.663475 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:31:04.663481 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:31:04.663533 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:31:04.670331 (+ 6798us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:31:04.671663 (+ 1332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:31:04.671669 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:31:04.671673 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:31:04.671880 (+ 207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:31:04.672526 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:31:04.672532 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:31:04.672535 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:31:04.672951 (+ 416us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:31:04.673195 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0810 11:31:04.674190 (+ 995us) server_negotiation.cc:299] Negotiation successful
0810 11:31:04.674439 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":967,"thread_start_us":776,"threads_started":1}
I20260810 11:31:04.676568 15598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:04.624824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.252:41545 (local address 127.8.18.193:35399)
0810 11:31:04.639324 (+ 14500us) negotiation.cc:107] Waiting for socket to connect
0810 11:31:04.639348 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0810 11:31:04.641915 (+ 2567us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:31:04.643257 (+ 1342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:31:04.643272 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:31:04.645140 (+ 1868us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:31:04.647776 (+ 2636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:04.647816 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:04.650084 (+ 2268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:04.650095 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:04.654588 (+ 4493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:04.654616 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:04.656862 (+ 2246us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:04.656870 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:04.657446 (+ 576us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:04.658811 (+ 1365us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:31:04.658848 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:31:04.662581 (+ 3733us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:31:04.667828 (+ 5247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:31:04.667840 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:31:04.667862 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:31:04.668762 (+ 900us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:31:04.669615 (+ 853us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:31:04.669622 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:31:04.669625 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:31:04.670143 (+ 518us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:31:04.671099 (+ 956us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:31:04.671108 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:31:04.671673 (+ 565us) client_negotiation.cc:769] Sending connection context
0810 11:31:04.673507 (+ 1834us) client_negotiation.cc:240] Negotiation successful
0810 11:31:04.674211 (+ 704us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14141,"mutex_wait_us":247,"spinlock_wait_cycles":4992,"thread_start_us":13558,"threads_started":1}
I20260810 11:31:04.677321 15602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:04.641007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.253:44341 (local address 127.8.18.193:51279)
0810 11:31:04.642022 (+ 1015us) negotiation.cc:107] Waiting for socket to connect
0810 11:31:04.642042 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 11:31:04.642302 (+ 260us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:31:04.643257 (+ 955us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:31:04.643272 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:31:04.645747 (+ 2475us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:31:04.647775 (+ 2028us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:04.647816 (+ 41us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:04.651774 (+ 3958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:04.651780 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:04.656747 (+ 4967us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:04.656764 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:04.659046 (+ 2282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:04.659055 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:04.659431 (+ 376us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:04.660114 (+ 683us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:31:04.660138 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:31:04.663124 (+ 2986us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:31:04.670521 (+ 7397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:31:04.670526 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:31:04.670529 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:31:04.671475 (+ 946us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:31:04.672124 (+ 649us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:31:04.672129 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:31:04.672133 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:31:04.672372 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:31:04.673633 (+ 1261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:31:04.673638 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:31:04.673841 (+ 203us) client_negotiation.cc:769] Sending connection context
0810 11:31:04.675891 (+ 2050us) client_negotiation.cc:240] Negotiation successful
0810 11:31:04.676475 (+ 584us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":827,"mutex_wait_us":103,"thread_start_us":626,"threads_started":1}
I20260810 11:31:04.684507 15593 heartbeater.cc:347] Connected to a master server at 127.8.18.253:44341
I20260810 11:31:04.684630 15591 heartbeater.cc:347] Connected to a master server at 127.8.18.252:41545
I20260810 11:31:04.685117 15591 heartbeater.cc:464] Registering TS with master...
I20260810 11:31:04.685117 15593 heartbeater.cc:464] Registering TS with master...
I20260810 11:31:04.687400 15593 heartbeater.cc:511] Master 127.8.18.253:44341 requested a full tablet report, sending...
I20260810 11:31:04.691735 15210 ts_manager.cc:194] Registered new tserver with Master: b087d7b7ed0f44b585cbb87fa59cda8b (127.8.18.193:44877)
I20260810 11:31:04.693130 15591 heartbeater.cc:511] Master 127.8.18.252:41545 requested a full tablet report, sending...
I20260810 11:31:04.694684 15597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:04.623572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:36523 (local address 127.8.18.193:50009)
0810 11:31:04.636157 (+ 12585us) negotiation.cc:107] Waiting for socket to connect
0810 11:31:04.636294 (+ 137us) client_negotiation.cc:174] Beginning negotiation
0810 11:31:04.641912 (+ 5618us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:31:04.643257 (+ 1345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:31:04.643272 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:31:04.645275 (+ 2003us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:31:04.648139 (+ 2864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:04.648153 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:04.651743 (+ 3590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:04.651750 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:04.654919 (+ 3169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:04.654949 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:04.656715 (+ 1766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:04.656727 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:04.657447 (+ 720us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:04.659385 (+ 1938us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:31:04.659429 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:31:04.663952 (+ 4523us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:31:04.685924 (+ 21972us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:31:04.685934 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:31:04.685941 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:31:04.687280 (+ 1339us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:31:04.688542 (+ 1262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:31:04.688552 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:31:04.688558 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:31:04.688890 (+ 332us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:31:04.690232 (+ 1342us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:31:04.690237 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:31:04.690588 (+ 351us) client_negotiation.cc:769] Sending connection context
0810 11:31:04.690832 (+ 244us) client_negotiation.cc:240] Negotiation successful
0810 11:31:04.693401 (+ 2569us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12037,"mutex_wait_us":106,"spinlock_wait_cycles":59648,"thread_start_us":11522,"threads_started":1}
I20260810 11:31:04.694811 15596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:04.622080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.193:50009 (local address 127.8.18.254:36523)
0810 11:31:04.623287 (+ 1207us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:04.623299 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:04.636721 (+ 13422us) server_negotiation.cc:407] Connection header received
0810 11:31:04.642348 (+ 5627us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:04.642357 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:04.642454 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:04.642545 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:31:04.649609 (+ 7064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:04.651501 (+ 1892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:04.655165 (+ 3664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:04.656257 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:04.656409 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:04.664631 (+ 8222us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:31:04.664641 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:31:04.664646 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:31:04.664695 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:31:04.685714 (+ 21019us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:31:04.687683 (+ 1969us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:31:04.687700 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:31:04.687714 (+ 14us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:31:04.688199 (+ 485us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:31:04.689101 (+ 902us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:31:04.689107 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:31:04.689111 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:31:04.689788 (+ 677us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:31:04.690122 (+ 334us) server_negotiation.cc:1035] Waiting for connection context
0810 11:31:04.691826 (+ 1704us) server_negotiation.cc:299] Negotiation successful
0810 11:31:04.692382 (+ 556us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":977,"thread_start_us":832,"threads_started":1}
I20260810 11:31:04.696557 15592 heartbeater.cc:347] Connected to a master server at 127.8.18.254:36523
I20260810 11:31:04.696791 15592 heartbeater.cc:464] Registering TS with master...
I20260810 11:31:04.697094 15406 ts_manager.cc:194] Registered new tserver with Master: b087d7b7ed0f44b585cbb87fa59cda8b (127.8.18.193:44877)
I20260810 11:31:04.697801 15592 heartbeater.cc:511] Master 127.8.18.254:36523 requested a full tablet report, sending...
I20260810 11:31:04.699735 14591 ts_manager.cc:194] Re-registered known tserver with Master: b087d7b7ed0f44b585cbb87fa59cda8b (127.8.18.193:44877)
I20260810 11:31:04.700907 14591 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.18.193@KRBTEST.COM'} at 127.8.18.193:50009
I20260810 11:31:04.784828 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 14794
I20260810 11:31:04.804961 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.18.194:40169
--local_ip_for_outbound_sockets=127.8.18.194
--tserver_master_addrs=127.8.18.254:36523
--webserver_port=34993
--webserver_interface=127.8.18.194
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/krb5kdc/oryx_127.8.18.194.keytab
--principal=oryx/127.8.18.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.18.212:34803
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.8.18.254:36523,127.8.18.253:44341
--tserver_master_addrs=127.8.18.254:36523,127.8.18.253:44341,127.8.18.252:41545 with env {KRB5CCNAME=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 11:31:04.892989 15605 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd8ee5823e444d28a6cb100898b494c5 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "cd8ee5823e444d28a6cb100898b494c5" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "cd8ee5823e444d28a6cb100898b494c5" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 36523 } } peers { permanent_uuid: "8fb82cee99b449c5a33adb02cba79748" member_type: VOTER last_known_addr { host: "127.8.18.253" port: 44341 } attrs { promote: false } } peers { permanent_uuid: "d1ea41131c2a4c6f984571036be55232" member_type: NON_VOTER last_known_addr { host: "127.8.18.252" port: 41545 } attrs { promote: true } } }
I20260810 11:31:04.893882 15605 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd8ee5823e444d28a6cb100898b494c5 [sys.catalog]: This master's current role is: LEADER
I20260810 11:31:04.894366 15607 raft_consensus.cc:1050] T 00000000000000000000000000000000 P cd8ee5823e444d28a6cb100898b494c5: attempting to promote NON_VOTER d1ea41131c2a4c6f984571036be55232 to VOTER
I20260810 11:31:04.895057 15607 consensus_queue.cc:237] T 00000000000000000000000000000000 P cd8ee5823e444d28a6cb100898b494c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "cd8ee5823e444d28a6cb100898b494c5" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 36523 } } peers { permanent_uuid: "8fb82cee99b449c5a33adb02cba79748" member_type: VOTER last_known_addr { host: "127.8.18.253" port: 44341 } attrs { promote: false } } peers { permanent_uuid: "d1ea41131c2a4c6f984571036be55232" member_type: VOTER last_known_addr { host: "127.8.18.252" port: 41545 } attrs { promote: false } }
I20260810 11:31:04.899466 15416 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d1ea41131c2a4c6f984571036be55232 [term 1 LEARNER]: Refusing update from remote peer cd8ee5823e444d28a6cb100898b494c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260810 11:31:04.900591 15220 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8fb82cee99b449c5a33adb02cba79748 [term 1 FOLLOWER]: Refusing update from remote peer cd8ee5823e444d28a6cb100898b494c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260810 11:31:04.901623 15612 consensus_queue.cc:1035] T 00000000000000000000000000000000 P cd8ee5823e444d28a6cb100898b494c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1ea41131c2a4c6f984571036be55232" member_type: VOTER last_known_addr { host: "127.8.18.252" port: 41545 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260810 11:31:04.902477 15605 consensus_queue.cc:1035] T 00000000000000000000000000000000 P cd8ee5823e444d28a6cb100898b494c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fb82cee99b449c5a33adb02cba79748" member_type: VOTER last_known_addr { host: "127.8.18.253" port: 44341 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260810 11:31:04.906752 15220 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 8fb82cee99b449c5a33adb02cba79748 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, d1ea41131c2a4c6f984571036be55232 (127.8.18.252) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "cd8ee5823e444d28a6cb100898b494c5" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 36523 } } peers { permanent_uuid: "8fb82cee99b449c5a33adb02cba79748" member_type: VOTER last_known_addr { host: "127.8.18.253" port: 44341 } attrs { promote: false } } peers { permanent_uuid: "d1ea41131c2a4c6f984571036be55232" member_type: VOTER last_known_addr { host: "127.8.18.252" port: 41545 } attrs { promote: false } } }
I20260810 11:31:04.905964 15612 raft_consensus.cc:2932] T 00000000000000000000000000000000 P cd8ee5823e444d28a6cb100898b494c5 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, d1ea41131c2a4c6f984571036be55232 (127.8.18.252) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "cd8ee5823e444d28a6cb100898b494c5" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 36523 } } peers { permanent_uuid: "8fb82cee99b449c5a33adb02cba79748" member_type: VOTER last_known_addr { host: "127.8.18.253" port: 44341 } attrs { promote: false } } peers { permanent_uuid: "d1ea41131c2a4c6f984571036be55232" member_type: VOTER last_known_addr { host: "127.8.18.252" port: 41545 } attrs { promote: false } } }
I20260810 11:31:04.914374 15607 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd8ee5823e444d28a6cb100898b494c5 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "cd8ee5823e444d28a6cb100898b494c5" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "cd8ee5823e444d28a6cb100898b494c5" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 36523 } } peers { permanent_uuid: "8fb82cee99b449c5a33adb02cba79748" member_type: VOTER last_known_addr { host: "127.8.18.253" port: 44341 } attrs { promote: false } } peers { permanent_uuid: "d1ea41131c2a4c6f984571036be55232" member_type: VOTER last_known_addr { host: "127.8.18.252" port: 41545 } attrs { promote: false } } }
I20260810 11:31:04.914856 15607 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd8ee5823e444d28a6cb100898b494c5 [sys.catalog]: This master's current role is: LEADER
I20260810 11:31:04.915475 15607 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd8ee5823e444d28a6cb100898b494c5 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "cd8ee5823e444d28a6cb100898b494c5" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "cd8ee5823e444d28a6cb100898b494c5" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 36523 } } peers { permanent_uuid: "8fb82cee99b449c5a33adb02cba79748" member_type: VOTER last_known_addr { host: "127.8.18.253" port: 44341 } attrs { promote: false } } peers { permanent_uuid: "d1ea41131c2a4c6f984571036be55232" member_type: VOTER last_known_addr { host: "127.8.18.252" port: 41545 } attrs { promote: false } } }
I20260810 11:31:04.915823 15607 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd8ee5823e444d28a6cb100898b494c5 [sys.catalog]: This master's current role is: LEADER
I20260810 11:31:04.917748 15615 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8fb82cee99b449c5a33adb02cba79748 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "cd8ee5823e444d28a6cb100898b494c5" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "cd8ee5823e444d28a6cb100898b494c5" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 36523 } } peers { permanent_uuid: "8fb82cee99b449c5a33adb02cba79748" member_type: VOTER last_known_addr { host: "127.8.18.253" port: 44341 } attrs { promote: false } } peers { permanent_uuid: "d1ea41131c2a4c6f984571036be55232" member_type: VOTER last_known_addr { host: "127.8.18.252" port: 41545 } attrs { promote: false } } }
I20260810 11:31:04.918499 15615 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8fb82cee99b449c5a33adb02cba79748 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 11:31:04.919543 15605 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd8ee5823e444d28a6cb100898b494c5 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "cd8ee5823e444d28a6cb100898b494c5" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "cd8ee5823e444d28a6cb100898b494c5" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 36523 } } peers { permanent_uuid: "8fb82cee99b449c5a33adb02cba79748" member_type: VOTER last_known_addr { host: "127.8.18.253" port: 44341 } attrs { promote: false } } peers { permanent_uuid: "d1ea41131c2a4c6f984571036be55232" member_type: VOTER last_known_addr { host: "127.8.18.252" port: 41545 } attrs { promote: false } } }
I20260810 11:31:04.919919 15605 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd8ee5823e444d28a6cb100898b494c5 [sys.catalog]: This master's current role is: LEADER
I20260810 11:31:04.921154 15416 raft_consensus.cc:2932] T 00000000000000000000000000000000 P d1ea41131c2a4c6f984571036be55232 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, d1ea41131c2a4c6f984571036be55232 (127.8.18.252) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "cd8ee5823e444d28a6cb100898b494c5" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 36523 } } peers { permanent_uuid: "8fb82cee99b449c5a33adb02cba79748" member_type: VOTER last_known_addr { host: "127.8.18.253" port: 44341 } attrs { promote: false } } peers { permanent_uuid: "d1ea41131c2a4c6f984571036be55232" member_type: VOTER last_known_addr { host: "127.8.18.252" port: 41545 } attrs { promote: false } } }
I20260810 11:31:04.927145 15616 sys_catalog.cc:455] T 00000000000000000000000000000000 P d1ea41131c2a4c6f984571036be55232 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "cd8ee5823e444d28a6cb100898b494c5" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "cd8ee5823e444d28a6cb100898b494c5" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 36523 } } peers { permanent_uuid: "8fb82cee99b449c5a33adb02cba79748" member_type: VOTER last_known_addr { host: "127.8.18.253" port: 44341 } attrs { promote: false } } peers { permanent_uuid: "d1ea41131c2a4c6f984571036be55232" member_type: VOTER last_known_addr { host: "127.8.18.252" port: 41545 } attrs { promote: false } } }
I20260810 11:31:04.927709 15616 sys_catalog.cc:458] T 00000000000000000000000000000000 P d1ea41131c2a4c6f984571036be55232 [sys.catalog]: This master's current role is: FOLLOWER
W20260810 11:31:05.308245 15604 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:31:05.308590 15604 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:31:05.308771 15604 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:31:05.317966 15604 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:31:05.318174 15604 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:31:05.318418 15604 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.18.194
I20260810 11:31:05.329731 15604 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.18.212:34803
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/krb5kdc/oryx_127.8.18.194.keytab
--principal=oryx/127.8.18.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.18.194:40169
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.18.194
--webserver_port=34993
--tserver_master_addrs=127.8.18.254:36523,127.8.18.253:44341,127.8.18.252:41545
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.18.194
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:31:05.331418 15604 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:31:05.333019 15604 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:31:05.349104 15624 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:31:05.349161 15622 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:31:05.349514 15621 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:31:05.350190 15604 server_base.cc:1034] running on GCE node
I20260810 11:31:05.350962 15604 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:31:05.352173 15604 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:31:05.353502 15604 hybrid_clock.cc:648] HybridClock initialized: now 1786361465353441 us; error 49 us; skew 500 ppm
I20260810 11:31:05.353932 15604 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:31:05 dist-test-slave-srgv krb5kdc[14545](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361465, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.18.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:31:05.357906 15604 init.cc:377] Logged in from keytab as oryx/127.8.18.194@KRBTEST.COM (short username oryx)
I20260810 11:31:05.359616 15604 webserver.cc:460] Webserver started at http://127.8.18.194:34993/ using document root <none> and password file <none>
I20260810 11:31:05.360414 15604 fs_manager.cc:359] Metadata directory not provided
I20260810 11:31:05.360560 15604 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:31:05.366946 15604 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260810 11:31:05.370744 15631 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:31:05.372627 15604 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260810 11:31:05.372844 15604 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/ts-1/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/ts-1/wal
uuid: "c19c62abc08e457eb6c66d5baff5682d"
format_stamp: "Formatted at 2026-08-10 11:31:01 on dist-test-slave-srgv"
server_key: "bb43c441f10977408bde5045d44813d2"
server_key_iv: "263df37826ac6621e36de7b0cfcb80ba"
server_key_version: "encryptionkey@0"
I20260810 11:31:05.374173 15604 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:31:05.397732 15604 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:31:05.403555 15604 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:31:05.403878 15604 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:31:05.405014 15604 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:31:05.406668 15604 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:31:05.406781 15604 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:31:05.406872 15604 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:31:05.406936 15604 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:31:05.445993 15604 rpc_server.cc:307] RPC server started. Bound to: 127.8.18.194:40169
I20260810 11:31:05.446165 15748 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.18.194:40169 every 8 connection(s)
I20260810 11:31:05.447793 15604 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/ts-1/data/info.pb
I20260810 11:31:05.456689 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 15604
I20260810 11:31:05.457080 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 14932
Aug 10 11:31:05 dist-test-slave-srgv krb5kdc[14545](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361459, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.8.18.195@KRBTEST.COM
Aug 10 11:31:05 dist-test-slave-srgv krb5kdc[14545](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361465, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.18.194@KRBTEST.COM for oryx/127.8.18.253@KRBTEST.COM
Aug 10 11:31:05 dist-test-slave-srgv krb5kdc[14545](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361465, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.18.194@KRBTEST.COM for oryx/127.8.18.252@KRBTEST.COM
Aug 10 11:31:05 dist-test-slave-srgv krb5kdc[14545](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361465, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.18.194@KRBTEST.COM for oryx/127.8.18.254@KRBTEST.COM
I20260810 11:31:05.497861 15760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:05.458925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41066 (local address 127.8.18.195:35115)
0810 11:31:05.465216 (+ 6291us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:05.465229 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:05.466781 (+ 1552us) server_negotiation.cc:407] Connection header received
0810 11:31:05.467039 (+ 258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:05.467052 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:05.467582 (+ 530us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:05.467809 (+ 227us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:31:05.471204 (+ 3395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:05.473564 (+ 2360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:05.476463 (+ 2899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:05.479525 (+ 3062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:05.479778 (+ 253us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:05.488413 (+ 8635us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:31:05.488436 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:31:05.488457 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:31:05.488507 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:31:05.493490 (+ 4983us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:31:05.494476 (+ 986us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:31:05.494490 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:31:05.494496 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:31:05.494698 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:31:05.495414 (+ 716us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:31:05.495422 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:31:05.495425 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:31:05.495952 (+ 527us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:31:05.496309 (+ 357us) server_negotiation.cc:1035] Waiting for connection context
0810 11:31:05.496746 (+ 437us) server_negotiation.cc:299] Negotiation successful
0810 11:31:05.497063 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6007,"thread_start_us":5776,"threads_started":1}
I20260810 11:31:05.499804 15059 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:31:05.507162 15753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:05.451635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.194:46635 (local address 127.8.18.253:44341)
0810 11:31:05.472910 (+ 21275us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:05.472920 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:05.472951 (+ 31us) server_negotiation.cc:407] Connection header received
0810 11:31:05.473040 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:05.473049 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:05.473175 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:05.476975 (+ 3800us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:31:05.477223 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:05.478881 (+ 1658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:05.482635 (+ 3754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:05.483097 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:05.483272 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:05.490122 (+ 6850us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:31:05.490130 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:31:05.490137 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:31:05.490201 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:31:05.495120 (+ 4919us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:31:05.497728 (+ 2608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:31:05.497737 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:31:05.497742 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:31:05.498801 (+ 1059us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:31:05.504394 (+ 5593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:31:05.504404 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:31:05.504411 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:31:05.504860 (+ 449us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:31:05.505011 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0810 11:31:05.506406 (+ 1395us) server_negotiation.cc:299] Negotiation successful
0810 11:31:05.506763 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21061,"thread_start_us":21737,"threads_started":1}
I20260810 11:31:05.507658 15757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:05.456875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.194:34613 (local address 127.8.18.252:41545)
0810 11:31:05.484970 (+ 28095us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:05.484979 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:05.485963 (+ 984us) server_negotiation.cc:407] Connection header received
0810 11:31:05.486114 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:05.486120 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:05.486225 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:05.486338 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:31:05.488690 (+ 2352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:05.489853 (+ 1163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:05.492133 (+ 2280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:05.492902 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:05.493052 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:05.498014 (+ 4962us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:31:05.498024 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:31:05.498031 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:31:05.498097 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:31:05.503172 (+ 5075us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:31:05.505088 (+ 1916us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:31:05.505099 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:31:05.505107 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:31:05.505253 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:31:05.505736 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:31:05.505743 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:31:05.505747 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:31:05.506175 (+ 428us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:31:05.506329 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0810 11:31:05.507020 (+ 691us) server_negotiation.cc:299] Negotiation successful
0810 11:31:05.507287 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27836,"thread_start_us":28178,"threads_started":1}
I20260810 11:31:05.508579 15754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:05.452555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.253:44341 (local address 127.8.18.194:46635)
0810 11:31:05.454571 (+ 2016us) negotiation.cc:107] Waiting for socket to connect
0810 11:31:05.454648 (+ 77us) client_negotiation.cc:174] Beginning negotiation
0810 11:31:05.460270 (+ 5622us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:31:05.474078 (+ 13808us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:31:05.474101 (+ 23us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:31:05.475328 (+ 1227us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:31:05.476623 (+ 1295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:05.476665 (+ 42us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:05.479237 (+ 2572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:05.479250 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:05.482373 (+ 3123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:05.482395 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:05.483506 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:05.483518 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:05.483990 (+ 472us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:05.485168 (+ 1178us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:31:05.485216 (+ 48us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:31:05.489735 (+ 4519us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:31:05.495331 (+ 5596us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:31:05.495341 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:31:05.495363 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:31:05.497482 (+ 2119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:31:05.498995 (+ 1513us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:31:05.499000 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:31:05.499004 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:31:05.504191 (+ 5187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:31:05.505174 (+ 983us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:31:05.505187 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:31:05.505798 (+ 611us) client_negotiation.cc:769] Sending connection context
0810 11:31:05.506199 (+ 401us) client_negotiation.cc:240] Negotiation successful
0810 11:31:05.506890 (+ 691us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":850,"mutex_wait_us":1,"spinlock_wait_cycles":1280,"thread_start_us":1876,"threads_started":1}
I20260810 11:31:05.508631 15756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:05.456306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.252:41545 (local address 127.8.18.194:34613)
0810 11:31:05.485819 (+ 29513us) negotiation.cc:107] Waiting for socket to connect
0810 11:31:05.485840 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 11:31:05.485988 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:31:05.486589 (+ 601us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:31:05.486597 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:31:05.488079 (+ 1482us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:31:05.488446 (+ 367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:05.488464 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:05.490042 (+ 1578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:05.490047 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:05.491876 (+ 1829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:05.491897 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:05.493359 (+ 1462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:05.493367 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:05.493639 (+ 272us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:05.494306 (+ 667us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:31:05.494338 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:31:05.497703 (+ 3365us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:31:05.503436 (+ 5733us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:31:05.503442 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:31:05.503447 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:31:05.504906 (+ 1459us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:31:05.505392 (+ 486us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:31:05.505397 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:31:05.505401 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:31:05.505596 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:31:05.506398 (+ 802us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:31:05.506402 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:31:05.506552 (+ 150us) client_negotiation.cc:769] Sending connection context
0810 11:31:05.506699 (+ 147us) client_negotiation.cc:240] Negotiation successful
0810 11:31:05.508032 (+ 1333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29008,"mutex_wait_us":1010,"thread_start_us":28923,"threads_started":1}
I20260810 11:31:05.509438 15755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:05.453810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.194:42599 (local address 127.8.18.254:36523)
0810 11:31:05.482225 (+ 28415us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:05.482234 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:05.484213 (+ 1979us) server_negotiation.cc:407] Connection header received
0810 11:31:05.484495 (+ 282us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:05.484506 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:05.484721 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:05.484874 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:31:05.487649 (+ 2775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:05.488977 (+ 1328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:05.491643 (+ 2666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:05.492327 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:05.492481 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:05.498677 (+ 6196us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:31:05.498684 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:31:05.498689 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:31:05.498736 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:31:05.503219 (+ 4483us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:31:05.505074 (+ 1855us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:31:05.505080 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:31:05.505084 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:31:05.505302 (+ 218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:31:05.506047 (+ 745us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:31:05.506054 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:31:05.506057 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:31:05.506442 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:31:05.506613 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0810 11:31:05.508125 (+ 1512us) server_negotiation.cc:299] Negotiation successful
0810 11:31:05.508419 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28262,"thread_start_us":28123,"threads_started":1}
I20260810 11:31:05.508800 15758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:05.454034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:36523 (local address 127.8.18.194:42599)
0810 11:31:05.484060 (+ 30026us) negotiation.cc:107] Waiting for socket to connect
0810 11:31:05.484082 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 11:31:05.484332 (+ 250us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:31:05.486173 (+ 1841us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:31:05.486177 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:31:05.486946 (+ 769us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:31:05.487422 (+ 476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:05.487434 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:05.489531 (+ 2097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:05.489539 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:05.491389 (+ 1850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:05.491408 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:05.492794 (+ 1386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:05.492802 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:05.492984 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:05.494552 (+ 1568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:31:05.494582 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:31:05.498359 (+ 3777us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:31:05.503948 (+ 5589us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:31:05.503957 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:31:05.503963 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:31:05.504907 (+ 944us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:31:05.505767 (+ 860us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:31:05.505771 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:31:05.505774 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:31:05.505913 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:31:05.507521 (+ 1608us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:31:05.507526 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:31:05.507716 (+ 190us) client_negotiation.cc:769] Sending connection context
0810 11:31:05.507972 (+ 256us) client_negotiation.cc:240] Negotiation successful
0810 11:31:05.508241 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29723,"thread_start_us":29458,"threads_started":1}
I20260810 11:31:05.510422 15751 heartbeater.cc:347] Connected to a master server at 127.8.18.253:44341
I20260810 11:31:05.510565 15749 heartbeater.cc:347] Connected to a master server at 127.8.18.252:41545
I20260810 11:31:05.510800 15749 heartbeater.cc:464] Registering TS with master...
I20260810 11:31:05.510807 15751 heartbeater.cc:464] Registering TS with master...
I20260810 11:31:05.510968 15750 heartbeater.cc:347] Connected to a master server at 127.8.18.254:36523
I20260810 11:31:05.511090 15750 heartbeater.cc:464] Registering TS with master...
I20260810 11:31:05.512001 15751 heartbeater.cc:511] Master 127.8.18.253:44341 requested a full tablet report, sending...
I20260810 11:31:05.512032 15750 heartbeater.cc:511] Master 127.8.18.254:36523 requested a full tablet report, sending...
I20260810 11:31:05.512086 15749 heartbeater.cc:511] Master 127.8.18.252:41545 requested a full tablet report, sending...
I20260810 11:31:05.514112 14591 ts_manager.cc:194] Re-registered known tserver with Master: c19c62abc08e457eb6c66d5baff5682d (127.8.18.194:40169)
I20260810 11:31:05.514261 15210 ts_manager.cc:194] Registered new tserver with Master: c19c62abc08e457eb6c66d5baff5682d (127.8.18.194:40169)
I20260810 11:31:05.514256 15406 ts_manager.cc:194] Registered new tserver with Master: c19c62abc08e457eb6c66d5baff5682d (127.8.18.194:40169)
I20260810 11:31:05.515529 14591 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.18.194@KRBTEST.COM'} at 127.8.18.194:42599
I20260810 11:31:05.594638 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 14932
I20260810 11:31:05.614267 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.18.195:35115
--local_ip_for_outbound_sockets=127.8.18.195
--tserver_master_addrs=127.8.18.254:36523
--webserver_port=42243
--webserver_interface=127.8.18.195
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/krb5kdc/oryx_127.8.18.195.keytab
--principal=oryx/127.8.18.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.18.212:34803
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.8.18.254:36523,127.8.18.253:44341
--tserver_master_addrs=127.8.18.254:36523,127.8.18.253:44341,127.8.18.252:41545 with env {KRB5CCNAME=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 11:31:05.704130 15592 heartbeater.cc:503] Master 127.8.18.254:36523 was elected leader, sending a full tablet report...
W20260810 11:31:05.990182 15762 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:31:05.990615 15762 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:31:05.990792 15762 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:31:05.999244 15762 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:31:05.999424 15762 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:31:05.999614 15762 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.18.195
I20260810 11:31:06.011643 15762 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.18.212:34803
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/krb5kdc/oryx_127.8.18.195.keytab
--principal=oryx/127.8.18.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.18.195:35115
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.18.195
--webserver_port=42243
--tserver_master_addrs=127.8.18.254:36523,127.8.18.253:44341,127.8.18.252:41545
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.18.195
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:31:06.013141 15762 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:31:06.014707 15762 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:31:06.029630 15767 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:31:06.030566 15768 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:31:06.031904 15770 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:31:06.032439 15762 server_base.cc:1034] running on GCE node
I20260810 11:31:06.033151 15762 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:31:06.034451 15762 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:31:06.035723 15762 hybrid_clock.cc:648] HybridClock initialized: now 1786361466035639 us; error 75 us; skew 500 ppm
I20260810 11:31:06.036110 15762 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:31:06 dist-test-slave-srgv krb5kdc[14545](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361466, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.18.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:31:06.040289 15762 init.cc:377] Logged in from keytab as oryx/127.8.18.195@KRBTEST.COM (short username oryx)
I20260810 11:31:06.042662 15762 webserver.cc:460] Webserver started at http://127.8.18.195:42243/ using document root <none> and password file <none>
I20260810 11:31:06.043785 15762 fs_manager.cc:359] Metadata directory not provided
I20260810 11:31:06.043967 15762 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:31:06.052512 15762 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.005s sys 0.000s
I20260810 11:31:06.056483 15777 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:31:06.058439 15762 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260810 11:31:06.058684 15762 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/ts-2/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/ts-2/wal
uuid: "7f6e17ac59cb474d830f9789a7a7f849"
format_stamp: "Formatted at 2026-08-10 11:31:01 on dist-test-slave-srgv"
server_key: "97c100d0530ccb90f0833cea0839c84d"
server_key_iv: "3d021d5c7c30b69d8dae27784ad8a613"
server_key_version: "encryptionkey@0"
I20260810 11:31:06.059850 15762 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:31:06.099776 15762 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:31:06.105080 15762 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:31:06.105453 15762 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:31:06.106640 15762 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:31:06.108371 15762 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:31:06.108501 15762 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:31:06.108623 15762 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:31:06.108702 15762 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:31:06.148425 15762 rpc_server.cc:307] RPC server started. Bound to: 127.8.18.195:35115
I20260810 11:31:06.148491 15892 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.18.195:35115 every 8 connection(s)
I20260810 11:31:06.150146 15762 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/ts-2/data/info.pb
I20260810 11:31:06.155246 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 15762
Aug 10 11:31:06 dist-test-slave-srgv krb5kdc[14545](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361466, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.18.195@KRBTEST.COM for oryx/127.8.18.254@KRBTEST.COM
Aug 10 11:31:06 dist-test-slave-srgv krb5kdc[14545](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361466, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.18.195@KRBTEST.COM for oryx/127.8.18.252@KRBTEST.COM
I20260810 11:31:06.214598 15898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:06.153905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.195:57915 (local address 127.8.18.254:36523)
0810 11:31:06.165242 (+ 11337us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:06.165252 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:06.170774 (+ 5522us) server_negotiation.cc:407] Connection header received
0810 11:31:06.179966 (+ 9192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:06.179973 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:06.180074 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:06.180193 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:31:06.182228 (+ 2035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:06.183473 (+ 1245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:06.189383 (+ 5910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:06.190111 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:06.190278 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:06.196342 (+ 6064us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:31:06.196351 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:31:06.196358 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:31:06.196437 (+ 79us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:31:06.200629 (+ 4192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:31:06.209999 (+ 9370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:31:06.210010 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:31:06.210016 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:31:06.210327 (+ 311us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:31:06.211635 (+ 1308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:31:06.211646 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:31:06.211652 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:31:06.212274 (+ 622us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:31:06.212470 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0810 11:31:06.213824 (+ 1354us) server_negotiation.cc:299] Negotiation successful
0810 11:31:06.214130 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11129,"thread_start_us":10964,"threads_started":1}
Aug 10 11:31:06 dist-test-slave-srgv krb5kdc[14545](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361466, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.18.195@KRBTEST.COM for oryx/127.8.18.253@KRBTEST.COM
I20260810 11:31:06.220276 15897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:06.153870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.195:54755 (local address 127.8.18.252:41545)
0810 11:31:06.168637 (+ 14767us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:06.168647 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:06.170038 (+ 1391us) server_negotiation.cc:407] Connection header received
0810 11:31:06.175967 (+ 5929us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:06.175978 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:06.176123 (+ 145us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:06.176386 (+ 263us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:31:06.182272 (+ 5886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:06.183473 (+ 1201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:06.189600 (+ 6127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:06.190291 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:06.190393 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:06.199674 (+ 9281us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:31:06.199684 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:31:06.199689 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:31:06.199756 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:31:06.205265 (+ 5509us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:31:06.213616 (+ 8351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:31:06.213627 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:31:06.213634 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:31:06.213916 (+ 282us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:31:06.215312 (+ 1396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:31:06.215322 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:31:06.215328 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:31:06.217010 (+ 1682us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:31:06.217285 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0810 11:31:06.219191 (+ 1906us) server_negotiation.cc:299] Negotiation successful
0810 11:31:06.219630 (+ 439us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14529,"thread_start_us":14798,"threads_started":1}
I20260810 11:31:06.225199 15899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:06.154542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.252:41545 (local address 127.8.18.195:54755)
0810 11:31:06.169704 (+ 15162us) negotiation.cc:107] Waiting for socket to connect
0810 11:31:06.169829 (+ 125us) client_negotiation.cc:174] Beginning negotiation
0810 11:31:06.175362 (+ 5533us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:31:06.177237 (+ 1875us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:31:06.177257 (+ 20us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:31:06.178894 (+ 1637us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:31:06.180518 (+ 1624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:06.180564 (+ 46us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:06.183766 (+ 3202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:06.183776 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:06.189392 (+ 5616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:06.189402 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:06.191656 (+ 2254us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:06.191665 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:06.193706 (+ 2041us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:06.194790 (+ 1084us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:31:06.194828 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:31:06.199243 (+ 4415us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:31:06.211031 (+ 11788us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:31:06.211041 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:31:06.211048 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:31:06.212063 (+ 1015us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:31:06.214229 (+ 2166us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:31:06.214238 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:31:06.214245 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:31:06.214507 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:31:06.217647 (+ 3140us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:31:06.217655 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:31:06.217984 (+ 329us) client_negotiation.cc:769] Sending connection context
0810 11:31:06.220721 (+ 2737us) client_negotiation.cc:240] Negotiation successful
0810 11:31:06.221295 (+ 574us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14635,"thread_start_us":14312,"threads_started":1}
I20260810 11:31:06.227849 15893 heartbeater.cc:347] Connected to a master server at 127.8.18.252:41545
I20260810 11:31:06.228356 15893 heartbeater.cc:464] Registering TS with master...
I20260810 11:31:06.229772 15893 heartbeater.cc:511] Master 127.8.18.252:41545 requested a full tablet report, sending...
I20260810 11:31:06.232798 15406 ts_manager.cc:194] Registered new tserver with Master: 7f6e17ac59cb474d830f9789a7a7f849 (127.8.18.195:35115)
I20260810 11:31:06.235212 15900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:06.154542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:36523 (local address 127.8.18.195:57915)
0810 11:31:06.170619 (+ 16077us) negotiation.cc:107] Waiting for socket to connect
0810 11:31:06.170643 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0810 11:31:06.179727 (+ 9084us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:31:06.180862 (+ 1135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:31:06.180870 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:31:06.181610 (+ 740us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:31:06.182014 (+ 404us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:06.182031 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:06.185543 (+ 3512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:06.185552 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:06.189156 (+ 3604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:06.189176 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:06.190819 (+ 1643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:06.190827 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:06.191076 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:06.191984 (+ 908us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:31:06.192022 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:31:06.195993 (+ 3971us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:31:06.201054 (+ 5061us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:31:06.202211 (+ 1157us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:31:06.202235 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:31:06.209751 (+ 7516us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:31:06.210987 (+ 1236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:31:06.210992 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:31:06.210996 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:31:06.211427 (+ 431us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:31:06.212461 (+ 1034us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:31:06.212470 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:31:06.213221 (+ 751us) client_negotiation.cc:769] Sending connection context
0810 11:31:06.213677 (+ 456us) client_negotiation.cc:240] Negotiation successful
0810 11:31:06.214326 (+ 649us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15890,"mutex_wait_us":1756,"thread_start_us":15498,"threads_started":1}
I20260810 11:31:06.236472 15894 heartbeater.cc:347] Connected to a master server at 127.8.18.254:36523
I20260810 11:31:06.236689 15894 heartbeater.cc:464] Registering TS with master...
I20260810 11:31:06.237375 15894 heartbeater.cc:511] Master 127.8.18.254:36523 requested a full tablet report, sending...
I20260810 11:31:06.238837 14591 ts_manager.cc:194] Re-registered known tserver with Master: 7f6e17ac59cb474d830f9789a7a7f849 (127.8.18.195:35115)
I20260810 11:31:06.240450 14591 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.18.195@KRBTEST.COM'} at 127.8.18.195:57915
I20260810 11:31:06.244968 15904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:06.171074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.253:44341 (local address 127.8.18.195:35697)
0810 11:31:06.194578 (+ 23504us) negotiation.cc:107] Waiting for socket to connect
0810 11:31:06.194605 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0810 11:31:06.194815 (+ 210us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:31:06.196296 (+ 1481us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:31:06.196301 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:31:06.197055 (+ 754us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:31:06.197548 (+ 493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:06.197574 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:06.199943 (+ 2369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:06.199950 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:06.201603 (+ 1653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:06.201623 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:06.202855 (+ 1232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:06.202862 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:06.203070 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:06.203754 (+ 684us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:31:06.203778 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:31:06.218276 (+ 14498us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:31:06.236893 (+ 18617us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:31:06.236901 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:31:06.236905 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:31:06.237737 (+ 832us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:31:06.241866 (+ 4129us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:31:06.241872 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:31:06.241876 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:31:06.242669 (+ 793us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:31:06.243463 (+ 794us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:31:06.243468 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:31:06.243658 (+ 190us) client_negotiation.cc:769] Sending connection context
0810 11:31:06.243799 (+ 141us) client_negotiation.cc:240] Negotiation successful
0810 11:31:06.244230 (+ 431us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23304,"thread_start_us":23117,"threads_started":1}
I20260810 11:31:06.246609 15903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:06.170993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.195:35697 (local address 127.8.18.253:44341)
0810 11:31:06.188450 (+ 17457us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:06.188457 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:06.194738 (+ 6281us) server_negotiation.cc:407] Connection header received
0810 11:31:06.194998 (+ 260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:06.195003 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:06.195109 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:06.195230 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:31:06.198351 (+ 3121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:06.199729 (+ 1378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:06.201877 (+ 2148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:06.202620 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:06.202806 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:06.218656 (+ 15850us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:31:06.218668 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:31:06.218677 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:31:06.218755 (+ 78us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:31:06.233885 (+ 15130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:31:06.241378 (+ 7493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:31:06.241394 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:31:06.241398 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:31:06.241680 (+ 282us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:31:06.242871 (+ 1191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:31:06.242876 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:31:06.242880 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:31:06.243301 (+ 421us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:31:06.245907 (+ 2606us) server_negotiation.cc:1035] Waiting for connection context
0810 11:31:06.246052 (+ 145us) server_negotiation.cc:299] Negotiation successful
0810 11:31:06.246276 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17277,"thread_start_us":17470,"threads_started":1}
I20260810 11:31:06.247535 15895 heartbeater.cc:347] Connected to a master server at 127.8.18.253:44341
I20260810 11:31:06.247737 15895 heartbeater.cc:464] Registering TS with master...
I20260810 11:31:06.248190 15895 heartbeater.cc:511] Master 127.8.18.253:44341 requested a full tablet report, sending...
I20260810 11:31:06.249640 15210 ts_manager.cc:194] Registered new tserver with Master: 7f6e17ac59cb474d830f9789a7a7f849 (127.8.18.195:35115)
I20260810 11:31:06.265141 15898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:06.204380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40478 (local address 127.8.18.254:36523)
0810 11:31:06.214942 (+ 10562us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:06.214947 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:06.241911 (+ 26964us) server_negotiation.cc:407] Connection header received
0810 11:31:06.242185 (+ 274us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:06.242191 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:06.242294 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:06.242403 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:31:06.249925 (+ 7522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:06.251073 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:06.253042 (+ 1969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:06.253750 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:06.253912 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:06.257638 (+ 3726us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:31:06.257646 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:31:06.257650 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:31:06.257699 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:31:06.261519 (+ 3820us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:31:06.262603 (+ 1084us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:31:06.262612 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:31:06.262615 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:31:06.262834 (+ 219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:31:06.263279 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:31:06.263286 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:31:06.263289 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:31:06.263723 (+ 434us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:31:06.264092 (+ 369us) server_negotiation.cc:1035] Waiting for connection context
0810 11:31:06.264384 (+ 292us) server_negotiation.cc:299] Negotiation successful
0810 11:31:06.264668 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10427,"thread_start_us":41249,"threads_started":1}
I20260810 11:31:06.269115 15907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:06.203775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47978 (local address 127.8.18.253:44341)
0810 11:31:06.237197 (+ 33422us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:06.237206 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:06.237232 (+ 26us) server_negotiation.cc:407] Connection header received
0810 11:31:06.237306 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:06.237315 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:06.237468 (+ 153us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:06.237623 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:31:06.242503 (+ 4880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:06.244395 (+ 1892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:06.251020 (+ 6625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:06.251800 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:06.251956 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:06.256523 (+ 4567us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:31:06.256534 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:31:06.256541 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:31:06.256600 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:31:06.261247 (+ 4647us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:31:06.263037 (+ 1790us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:31:06.263056 (+ 19us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:31:06.263060 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:31:06.265535 (+ 2475us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:31:06.266754 (+ 1219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:31:06.266761 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:31:06.266765 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:31:06.267188 (+ 423us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:31:06.267492 (+ 304us) server_negotiation.cc:1035] Waiting for connection context
0810 11:31:06.268279 (+ 787us) server_negotiation.cc:299] Negotiation successful
0810 11:31:06.268764 (+ 485us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33195,"thread_start_us":30697,"threads_started":1}
I20260810 11:31:06.270517 15897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:06.232082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45128 (local address 127.8.18.252:41545)
0810 11:31:06.232313 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:06.232323 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:06.250219 (+ 17896us) server_negotiation.cc:407] Connection header received
0810 11:31:06.250300 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:06.250305 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:06.250405 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:06.250528 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:31:06.253143 (+ 2615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:06.254340 (+ 1197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:06.256734 (+ 2394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:06.257330 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:06.257558 (+ 228us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:06.261549 (+ 3991us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:31:06.261556 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:31:06.261563 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:31:06.261619 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:31:06.265256 (+ 3637us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:31:06.267051 (+ 1795us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:31:06.267058 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:31:06.267061 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:31:06.267264 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:31:06.267726 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:31:06.267732 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:31:06.267735 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:31:06.268176 (+ 441us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:31:06.268464 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0810 11:31:06.269850 (+ 1386us) server_negotiation.cc:299] Negotiation successful
0810 11:31:06.270121 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260810 11:31:06.270552 14591 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40478:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 11:31:06.278676 14591 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 11:31:06.328923 15923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:06.307618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.195:35115 (local address 127.0.0.1:35360)
0810 11:31:06.320809 (+ 13191us) negotiation.cc:107] Waiting for socket to connect
0810 11:31:06.320920 (+ 111us) client_negotiation.cc:174] Beginning negotiation
0810 11:31:06.321057 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:31:06.321739 (+ 682us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:31:06.321745 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:31:06.321776 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:31:06.322221 (+ 445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:06.322235 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:06.324414 (+ 2179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:06.324422 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:06.326532 (+ 2110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:06.326547 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:06.328004 (+ 1457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:06.328010 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:06.328195 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:06.328243 (+ 48us) client_negotiation.cc:769] Sending connection context
0810 11:31:06.328345 (+ 102us) client_negotiation.cc:240] Negotiation successful
0810 11:31:06.328499 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13080,"mutex_wait_us":2,"thread_start_us":10823,"threads_started":1}
I20260810 11:31:06.328923 15926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:06.304843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35360 (local address 127.8.18.195:35115)
0810 11:31:06.317767 (+ 12924us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:06.317778 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:06.321074 (+ 3296us) server_negotiation.cc:407] Connection header received
0810 11:31:06.321211 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:06.321219 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:06.321492 (+ 273us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:06.321686 (+ 194us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:31:06.322824 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:06.324268 (+ 1444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:06.326726 (+ 2458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:06.327797 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:06.327973 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:06.328234 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0810 11:31:06.328401 (+ 167us) server_negotiation.cc:299] Negotiation successful
0810 11:31:06.328582 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12703,"thread_start_us":12491,"threads_started":1}
I20260810 11:31:06.331032 15927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:06.305000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47022 (local address 127.8.18.194:40169)
0810 11:31:06.317136 (+ 12136us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:06.317149 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:06.320411 (+ 3262us) server_negotiation.cc:407] Connection header received
0810 11:31:06.320617 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:06.320629 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:06.320994 (+ 365us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:06.321245 (+ 251us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:31:06.322647 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:06.324090 (+ 1443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:06.327531 (+ 3441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:06.329489 (+ 1958us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:06.329606 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:06.329833 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0810 11:31:06.330477 (+ 644us) server_negotiation.cc:299] Negotiation successful
0810 11:31:06.330689 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11911,"thread_start_us":12149,"threads_started":1}
I20260810 11:31:06.331525 15928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:06.303548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.193:44877 (local address 127.0.0.1:44682)
0810 11:31:06.319317 (+ 15769us) negotiation.cc:107] Waiting for socket to connect
0810 11:31:06.319358 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0810 11:31:06.319626 (+ 268us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:31:06.323366 (+ 3740us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:31:06.323371 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:31:06.323399 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:31:06.323775 (+ 376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:06.323789 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:06.325741 (+ 1952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:06.325748 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:06.328124 (+ 2376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:06.328145 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:06.330516 (+ 2371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:06.330522 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:06.330733 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:06.330767 (+ 34us) client_negotiation.cc:769] Sending connection context
0810 11:31:06.330841 (+ 74us) client_negotiation.cc:240] Negotiation successful
0810 11:31:06.330989 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15227,"mutex_wait_us":1,"thread_start_us":17009,"threads_started":1}
I20260810 11:31:06.331961 15924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:06.304092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.194:40169 (local address 127.0.0.1:47022)
0810 11:31:06.320230 (+ 16138us) negotiation.cc:107] Waiting for socket to connect
0810 11:31:06.320257 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0810 11:31:06.320427 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:31:06.321951 (+ 1524us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:31:06.321955 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:31:06.321977 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:31:06.322444 (+ 467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:06.322456 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:06.324622 (+ 2166us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:06.324633 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:06.327307 (+ 2674us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:06.327324 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:06.330059 (+ 2735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:06.330067 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:06.330242 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:06.330281 (+ 39us) client_negotiation.cc:769] Sending connection context
0810 11:31:06.331386 (+ 1105us) client_negotiation.cc:240] Negotiation successful
0810 11:31:06.331569 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16004,"mutex_wait_us":1,"thread_start_us":15823,"threads_started":1}
I20260810 11:31:06.332653 15925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:06.304182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44682 (local address 127.8.18.193:44877)
0810 11:31:06.317149 (+ 12967us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:06.317157 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:06.322498 (+ 5341us) server_negotiation.cc:407] Connection header received
0810 11:31:06.322637 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:06.322645 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:06.323120 (+ 475us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:06.323343 (+ 223us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:31:06.323998 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:06.325555 (+ 1557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:06.328387 (+ 2832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:06.330231 (+ 1844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:06.331795 (+ 1564us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:06.332065 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0810 11:31:06.332152 (+ 87us) server_negotiation.cc:299] Negotiation successful
0810 11:31:06.332348 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12731,"thread_start_us":12650,"threads_started":1}
I20260810 11:31:06.333248 15826 tablet_service.cc:1467] Processing CreateTablet for tablet 513fad3711e744b48f29af9cf186e126 (DEFAULT_TABLE table=test-table [id=5a346598604d4e04a539c1922211eb1c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:31:06.335048 15826 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 513fad3711e744b48f29af9cf186e126. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:31:06.335868 15681 tablet_service.cc:1467] Processing CreateTablet for tablet 513fad3711e744b48f29af9cf186e126 (DEFAULT_TABLE table=test-table [id=5a346598604d4e04a539c1922211eb1c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:31:06.336304 15524 tablet_service.cc:1467] Processing CreateTablet for tablet 513fad3711e744b48f29af9cf186e126 (DEFAULT_TABLE table=test-table [id=5a346598604d4e04a539c1922211eb1c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:31:06.337324 15681 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 513fad3711e744b48f29af9cf186e126. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:31:06.337666 15524 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 513fad3711e744b48f29af9cf186e126. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:31:06.351975 15929 tablet_bootstrap.cc:492] T 513fad3711e744b48f29af9cf186e126 P c19c62abc08e457eb6c66d5baff5682d: Bootstrap starting.
I20260810 11:31:06.356065 15929 tablet_bootstrap.cc:654] T 513fad3711e744b48f29af9cf186e126 P c19c62abc08e457eb6c66d5baff5682d: Neither blocks nor log segments found. Creating new log.
I20260810 11:31:06.357546 15929 log.cc:824] T 513fad3711e744b48f29af9cf186e126 P c19c62abc08e457eb6c66d5baff5682d: Log is configured to *not* fsync() on all Append() calls
I20260810 11:31:06.358726 15930 tablet_bootstrap.cc:492] T 513fad3711e744b48f29af9cf186e126 P 7f6e17ac59cb474d830f9789a7a7f849: Bootstrap starting.
I20260810 11:31:06.361696 15929 tablet_bootstrap.cc:492] T 513fad3711e744b48f29af9cf186e126 P c19c62abc08e457eb6c66d5baff5682d: No bootstrap required, opened a new log
I20260810 11:31:06.362183 15929 ts_tablet_manager.cc:1397] T 513fad3711e744b48f29af9cf186e126 P c19c62abc08e457eb6c66d5baff5682d: Time spent bootstrapping tablet: real 0.010s user 0.009s sys 0.000s
I20260810 11:31:06.362147 15931 tablet_bootstrap.cc:492] T 513fad3711e744b48f29af9cf186e126 P b087d7b7ed0f44b585cbb87fa59cda8b: Bootstrap starting.
I20260810 11:31:06.363618 15930 tablet_bootstrap.cc:654] T 513fad3711e744b48f29af9cf186e126 P 7f6e17ac59cb474d830f9789a7a7f849: Neither blocks nor log segments found. Creating new log.
I20260810 11:31:06.364912 15930 log.cc:824] T 513fad3711e744b48f29af9cf186e126 P 7f6e17ac59cb474d830f9789a7a7f849: Log is configured to *not* fsync() on all Append() calls
I20260810 11:31:06.366417 15931 tablet_bootstrap.cc:654] T 513fad3711e744b48f29af9cf186e126 P b087d7b7ed0f44b585cbb87fa59cda8b: Neither blocks nor log segments found. Creating new log.
I20260810 11:31:06.367849 15931 log.cc:824] T 513fad3711e744b48f29af9cf186e126 P b087d7b7ed0f44b585cbb87fa59cda8b: Log is configured to *not* fsync() on all Append() calls
I20260810 11:31:06.368284 15930 tablet_bootstrap.cc:492] T 513fad3711e744b48f29af9cf186e126 P 7f6e17ac59cb474d830f9789a7a7f849: No bootstrap required, opened a new log
I20260810 11:31:06.368597 15930 ts_tablet_manager.cc:1397] T 513fad3711e744b48f29af9cf186e126 P 7f6e17ac59cb474d830f9789a7a7f849: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.005s
I20260810 11:31:06.377843 15929 raft_consensus.cc:348] T 513fad3711e744b48f29af9cf186e126 P c19c62abc08e457eb6c66d5baff5682d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b087d7b7ed0f44b585cbb87fa59cda8b" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 44877 } } peers { permanent_uuid: "c19c62abc08e457eb6c66d5baff5682d" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 40169 } } peers { permanent_uuid: "7f6e17ac59cb474d830f9789a7a7f849" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 35115 } }
I20260810 11:31:06.379089 15929 raft_consensus.cc:374] T 513fad3711e744b48f29af9cf186e126 P c19c62abc08e457eb6c66d5baff5682d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:31:06.379326 15929 raft_consensus.cc:729] T 513fad3711e744b48f29af9cf186e126 P c19c62abc08e457eb6c66d5baff5682d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c19c62abc08e457eb6c66d5baff5682d, State: Initialized, Role: FOLLOWER
I20260810 11:31:06.380610 15929 consensus_queue.cc:260] T 513fad3711e744b48f29af9cf186e126 P c19c62abc08e457eb6c66d5baff5682d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b087d7b7ed0f44b585cbb87fa59cda8b" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 44877 } } peers { permanent_uuid: "c19c62abc08e457eb6c66d5baff5682d" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 40169 } } peers { permanent_uuid: "7f6e17ac59cb474d830f9789a7a7f849" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 35115 } }
I20260810 11:31:06.386791 15930 raft_consensus.cc:348] T 513fad3711e744b48f29af9cf186e126 P 7f6e17ac59cb474d830f9789a7a7f849 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b087d7b7ed0f44b585cbb87fa59cda8b" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 44877 } } peers { permanent_uuid: "c19c62abc08e457eb6c66d5baff5682d" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 40169 } } peers { permanent_uuid: "7f6e17ac59cb474d830f9789a7a7f849" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 35115 } }
I20260810 11:31:06.387573 15930 raft_consensus.cc:374] T 513fad3711e744b48f29af9cf186e126 P 7f6e17ac59cb474d830f9789a7a7f849 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:31:06.387780 15930 raft_consensus.cc:729] T 513fad3711e744b48f29af9cf186e126 P 7f6e17ac59cb474d830f9789a7a7f849 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f6e17ac59cb474d830f9789a7a7f849, State: Initialized, Role: FOLLOWER
I20260810 11:31:06.388212 15931 tablet_bootstrap.cc:492] T 513fad3711e744b48f29af9cf186e126 P b087d7b7ed0f44b585cbb87fa59cda8b: No bootstrap required, opened a new log
I20260810 11:31:06.388823 15931 ts_tablet_manager.cc:1397] T 513fad3711e744b48f29af9cf186e126 P b087d7b7ed0f44b585cbb87fa59cda8b: Time spent bootstrapping tablet: real 0.027s user 0.009s sys 0.010s
I20260810 11:31:06.388849 15930 consensus_queue.cc:260] T 513fad3711e744b48f29af9cf186e126 P 7f6e17ac59cb474d830f9789a7a7f849 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b087d7b7ed0f44b585cbb87fa59cda8b" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 44877 } } peers { permanent_uuid: "c19c62abc08e457eb6c66d5baff5682d" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 40169 } } peers { permanent_uuid: "7f6e17ac59cb474d830f9789a7a7f849" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 35115 } }
I20260810 11:31:06.392252 15750 heartbeater.cc:503] Master 127.8.18.254:36523 was elected leader, sending a full tablet report...
I20260810 11:31:06.392886 15929 ts_tablet_manager.cc:1428] T 513fad3711e744b48f29af9cf186e126 P c19c62abc08e457eb6c66d5baff5682d: Time spent starting tablet: real 0.030s user 0.010s sys 0.012s
I20260810 11:31:06.409248 15936 raft_consensus.cc:482] T 513fad3711e744b48f29af9cf186e126 P c19c62abc08e457eb6c66d5baff5682d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 11:31:06.409809 15936 raft_consensus.cc:504] T 513fad3711e744b48f29af9cf186e126 P c19c62abc08e457eb6c66d5baff5682d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b087d7b7ed0f44b585cbb87fa59cda8b" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 44877 } } peers { permanent_uuid: "c19c62abc08e457eb6c66d5baff5682d" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 40169 } } peers { permanent_uuid: "7f6e17ac59cb474d830f9789a7a7f849" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 35115 } }
I20260810 11:31:06.412832 15894 heartbeater.cc:503] Master 127.8.18.254:36523 was elected leader, sending a full tablet report...
I20260810 11:31:06.414132 15930 ts_tablet_manager.cc:1428] T 513fad3711e744b48f29af9cf186e126 P 7f6e17ac59cb474d830f9789a7a7f849: Time spent starting tablet: real 0.045s user 0.029s sys 0.011s
I20260810 11:31:06.418974 15931 raft_consensus.cc:348] T 513fad3711e744b48f29af9cf186e126 P b087d7b7ed0f44b585cbb87fa59cda8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b087d7b7ed0f44b585cbb87fa59cda8b" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 44877 } } peers { permanent_uuid: "c19c62abc08e457eb6c66d5baff5682d" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 40169 } } peers { permanent_uuid: "7f6e17ac59cb474d830f9789a7a7f849" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 35115 } }
I20260810 11:31:06.419868 15931 raft_consensus.cc:374] T 513fad3711e744b48f29af9cf186e126 P b087d7b7ed0f44b585cbb87fa59cda8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:31:06.420045 15931 raft_consensus.cc:729] T 513fad3711e744b48f29af9cf186e126 P b087d7b7ed0f44b585cbb87fa59cda8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b087d7b7ed0f44b585cbb87fa59cda8b, State: Initialized, Role: FOLLOWER
I20260810 11:31:06.421943 15931 consensus_queue.cc:260] T 513fad3711e744b48f29af9cf186e126 P b087d7b7ed0f44b585cbb87fa59cda8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b087d7b7ed0f44b585cbb87fa59cda8b" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 44877 } } peers { permanent_uuid: "c19c62abc08e457eb6c66d5baff5682d" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 40169 } } peers { permanent_uuid: "7f6e17ac59cb474d830f9789a7a7f849" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 35115 } }
I20260810 11:31:06.425119 15936 leader_election.cc:288] T 513fad3711e744b48f29af9cf186e126 P c19c62abc08e457eb6c66d5baff5682d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b087d7b7ed0f44b585cbb87fa59cda8b (127.8.18.193:44877), 7f6e17ac59cb474d830f9789a7a7f849 (127.8.18.195:35115)
I20260810 11:31:06.429023 15931 ts_tablet_manager.cc:1428] T 513fad3711e744b48f29af9cf186e126 P b087d7b7ed0f44b585cbb87fa59cda8b: Time spent starting tablet: real 0.040s user 0.011s sys 0.016s
I20260810 11:31:06.438431 15925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:06.421575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.194:44019 (local address 127.8.18.193:44877)
0810 11:31:06.421933 (+ 358us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:06.421940 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:06.425081 (+ 3141us) server_negotiation.cc:407] Connection header received
0810 11:31:06.425741 (+ 660us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:06.425751 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:06.425926 (+ 175us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:06.426015 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:31:06.428579 (+ 2564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:06.429975 (+ 1396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:06.434801 (+ 4826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:06.436767 (+ 1966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:06.436908 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:06.436967 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0810 11:31:06.437885 (+ 918us) server_negotiation.cc:299] Negotiation successful
0810 11:31:06.438085 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138}
I20260810 11:31:06.438530 15938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:06.421380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.193:44877 (local address 127.8.18.194:44019)
0810 11:31:06.424891 (+ 3511us) negotiation.cc:107] Waiting for socket to connect
0810 11:31:06.424922 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0810 11:31:06.425315 (+ 393us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:31:06.426328 (+ 1013us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:31:06.426336 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:31:06.426399 (+ 63us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:31:06.428230 (+ 1831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:06.428253 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:06.430483 (+ 2230us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:06.430492 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:06.434546 (+ 4054us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:06.434566 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:06.437362 (+ 2796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:06.437367 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:06.437656 (+ 289us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:06.437689 (+ 33us) client_negotiation.cc:769] Sending connection context
0810 11:31:06.437779 (+ 90us) client_negotiation.cc:240] Negotiation successful
0810 11:31:06.437908 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3254,"thread_start_us":2856,"threads_started":1}
I20260810 11:31:06.440250 15546 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "513fad3711e744b48f29af9cf186e126" candidate_uuid: "c19c62abc08e457eb6c66d5baff5682d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b087d7b7ed0f44b585cbb87fa59cda8b" is_pre_election: true
I20260810 11:31:06.441008 15546 raft_consensus.cc:2454] T 513fad3711e744b48f29af9cf186e126 P b087d7b7ed0f44b585cbb87fa59cda8b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c19c62abc08e457eb6c66d5baff5682d in term 0.
I20260810 11:31:06.442096 15632 leader_election.cc:302] T 513fad3711e744b48f29af9cf186e126 P c19c62abc08e457eb6c66d5baff5682d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b087d7b7ed0f44b585cbb87fa59cda8b, c19c62abc08e457eb6c66d5baff5682d; no voters:
I20260810 11:31:06.442788 15936 raft_consensus.cc:2781] T 513fad3711e744b48f29af9cf186e126 P c19c62abc08e457eb6c66d5baff5682d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 11:31:06.443096 15936 raft_consensus.cc:482] T 513fad3711e744b48f29af9cf186e126 P c19c62abc08e457eb6c66d5baff5682d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 11:31:06.443383 15936 raft_consensus.cc:3037] T 513fad3711e744b48f29af9cf186e126 P c19c62abc08e457eb6c66d5baff5682d [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:31:06.443513 15926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:06.426322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.194:56003 (local address 127.8.18.195:35115)
0810 11:31:06.426633 (+ 311us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:06.426643 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:06.431466 (+ 4823us) server_negotiation.cc:407] Connection header received
0810 11:31:06.433167 (+ 1701us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:06.433176 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:06.433277 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:06.433390 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:31:06.435104 (+ 1714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:06.436745 (+ 1641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:06.440576 (+ 3831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:06.442109 (+ 1533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:06.442368 (+ 259us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:06.442467 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0810 11:31:06.443057 (+ 590us) server_negotiation.cc:299] Negotiation successful
0810 11:31:06.443221 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260810 11:31:06.443828 15940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:06.425345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.195:35115 (local address 127.8.18.194:56003)
0810 11:31:06.431337 (+ 5992us) negotiation.cc:107] Waiting for socket to connect
0810 11:31:06.431360 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0810 11:31:06.432961 (+ 1601us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:31:06.433661 (+ 700us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:31:06.433666 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:31:06.433695 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:31:06.434841 (+ 1146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:06.434858 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:06.436961 (+ 2103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:06.436967 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:06.440351 (+ 3384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:06.440373 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:06.442472 (+ 2099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:06.442482 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:06.442808 (+ 326us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:06.442854 (+ 46us) client_negotiation.cc:769] Sending connection context
0810 11:31:06.442962 (+ 108us) client_negotiation.cc:240] Negotiation successful
0810 11:31:06.443199 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5827,"thread_start_us":5508,"threads_started":1}
I20260810 11:31:06.445510 15848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "513fad3711e744b48f29af9cf186e126" candidate_uuid: "c19c62abc08e457eb6c66d5baff5682d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f6e17ac59cb474d830f9789a7a7f849" is_pre_election: true
I20260810 11:31:06.446336 15848 raft_consensus.cc:2454] T 513fad3711e744b48f29af9cf186e126 P 7f6e17ac59cb474d830f9789a7a7f849 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c19c62abc08e457eb6c66d5baff5682d in term 0.
I20260810 11:31:06.450070 15936 raft_consensus.cc:504] T 513fad3711e744b48f29af9cf186e126 P c19c62abc08e457eb6c66d5baff5682d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b087d7b7ed0f44b585cbb87fa59cda8b" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 44877 } } peers { permanent_uuid: "c19c62abc08e457eb6c66d5baff5682d" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 40169 } } peers { permanent_uuid: "7f6e17ac59cb474d830f9789a7a7f849" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 35115 } }
I20260810 11:31:06.451452 15936 leader_election.cc:288] T 513fad3711e744b48f29af9cf186e126 P c19c62abc08e457eb6c66d5baff5682d [CANDIDATE]: Term 1 election: Requested vote from peers b087d7b7ed0f44b585cbb87fa59cda8b (127.8.18.193:44877), 7f6e17ac59cb474d830f9789a7a7f849 (127.8.18.195:35115)
I20260810 11:31:06.452162 15546 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "513fad3711e744b48f29af9cf186e126" candidate_uuid: "c19c62abc08e457eb6c66d5baff5682d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b087d7b7ed0f44b585cbb87fa59cda8b"
I20260810 11:31:06.452504 15546 raft_consensus.cc:3037] T 513fad3711e744b48f29af9cf186e126 P b087d7b7ed0f44b585cbb87fa59cda8b [term 0 FOLLOWER]: Advancing to term 1
W20260810 11:31:06.453011 15752 tablet.cc:2369] T 513fad3711e744b48f29af9cf186e126 P c19c62abc08e457eb6c66d5baff5682d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 11:31:06.457001 15546 raft_consensus.cc:2454] T 513fad3711e744b48f29af9cf186e126 P b087d7b7ed0f44b585cbb87fa59cda8b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c19c62abc08e457eb6c66d5baff5682d in term 1.
I20260810 11:31:06.457753 15632 leader_election.cc:302] T 513fad3711e744b48f29af9cf186e126 P c19c62abc08e457eb6c66d5baff5682d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b087d7b7ed0f44b585cbb87fa59cda8b, c19c62abc08e457eb6c66d5baff5682d; no voters:
I20260810 11:31:06.458335 15936 raft_consensus.cc:2781] T 513fad3711e744b48f29af9cf186e126 P c19c62abc08e457eb6c66d5baff5682d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:31:06.458364 15848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "513fad3711e744b48f29af9cf186e126" candidate_uuid: "c19c62abc08e457eb6c66d5baff5682d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f6e17ac59cb474d830f9789a7a7f849"
I20260810 11:31:06.458700 15848 raft_consensus.cc:3037] T 513fad3711e744b48f29af9cf186e126 P 7f6e17ac59cb474d830f9789a7a7f849 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:31:06.458747 15936 raft_consensus.cc:686] T 513fad3711e744b48f29af9cf186e126 P c19c62abc08e457eb6c66d5baff5682d [term 1 LEADER]: Becoming Leader. State: Replica: c19c62abc08e457eb6c66d5baff5682d, State: Running, Role: LEADER
I20260810 11:31:06.459807 15936 consensus_queue.cc:237] T 513fad3711e744b48f29af9cf186e126 P c19c62abc08e457eb6c66d5baff5682d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b087d7b7ed0f44b585cbb87fa59cda8b" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 44877 } } peers { permanent_uuid: "c19c62abc08e457eb6c66d5baff5682d" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 40169 } } peers { permanent_uuid: "7f6e17ac59cb474d830f9789a7a7f849" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 35115 } }
I20260810 11:31:06.462544 15848 raft_consensus.cc:2454] T 513fad3711e744b48f29af9cf186e126 P 7f6e17ac59cb474d830f9789a7a7f849 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c19c62abc08e457eb6c66d5baff5682d in term 1.
I20260810 11:31:06.466624 14591 catalog_manager.cc:5495] T 513fad3711e744b48f29af9cf186e126 P c19c62abc08e457eb6c66d5baff5682d reported cstate change: term changed from 0 to 1, leader changed from <none> to c19c62abc08e457eb6c66d5baff5682d (127.8.18.194). New cstate: current_term: 1 leader_uuid: "c19c62abc08e457eb6c66d5baff5682d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b087d7b7ed0f44b585cbb87fa59cda8b" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 44877 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c19c62abc08e457eb6c66d5baff5682d" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 40169 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7f6e17ac59cb474d830f9789a7a7f849" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 35115 } health_report { overall_health: UNKNOWN } } }
I20260810 11:31:06.494089 15927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:06.486355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47034 (local address 127.8.18.194:40169)
0810 11:31:06.486547 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:06.486551 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:06.487087 (+ 536us) server_negotiation.cc:407] Connection header received
0810 11:31:06.487302 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:06.487306 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:06.487441 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:06.487581 (+ 140us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 11:31:06.488066 (+ 485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:06.489330 (+ 1264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:06.491249 (+ 1919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:06.491863 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:06.491966 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:06.492654 (+ 688us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 11:31:06.493213 (+ 559us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 11:31:06.493327 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0810 11:31:06.493504 (+ 177us) server_negotiation.cc:299] Negotiation successful
0810 11:31:06.493603 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260810 11:31:06.506382 15546 raft_consensus.cc:1261] T 513fad3711e744b48f29af9cf186e126 P b087d7b7ed0f44b585cbb87fa59cda8b [term 1 FOLLOWER]: Refusing update from remote peer c19c62abc08e457eb6c66d5baff5682d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 11:31:06.506388 15848 raft_consensus.cc:1261] T 513fad3711e744b48f29af9cf186e126 P 7f6e17ac59cb474d830f9789a7a7f849 [term 1 FOLLOWER]: Refusing update from remote peer c19c62abc08e457eb6c66d5baff5682d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 11:31:06.507953 15936 consensus_queue.cc:1035] T 513fad3711e744b48f29af9cf186e126 P c19c62abc08e457eb6c66d5baff5682d [LEADER]: Connected to new peer: Peer: permanent_uuid: "7f6e17ac59cb474d830f9789a7a7f849" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 11:31:06.508531 15935 consensus_queue.cc:1035] T 513fad3711e744b48f29af9cf186e126 P c19c62abc08e457eb6c66d5baff5682d [LEADER]: Connected to new peer: Peer: permanent_uuid: "b087d7b7ed0f44b585cbb87fa59cda8b" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 44877 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 11:31:06.560817 14590 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40478:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 11:31:06.561733 14590 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40478:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 11:31:06.571797 14590 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cd8ee5823e444d28a6cb100898b494c5: Sending DeleteTablet for 3 replicas of tablet 513fad3711e744b48f29af9cf186e126
I20260810 11:31:06.573652 15524 tablet_service.cc:1514] Processing DeleteTablet for tablet 513fad3711e744b48f29af9cf186e126 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:31:06 UTC) from {username='oryx', principal='oryx/127.8.18.254@KRBTEST.COM'} at 127.0.0.1:44682
I20260810 11:31:06.573830 15681 tablet_service.cc:1514] Processing DeleteTablet for tablet 513fad3711e744b48f29af9cf186e126 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:31:06 UTC) from {username='oryx', principal='oryx/127.8.18.254@KRBTEST.COM'} at 127.0.0.1:47022
I20260810 11:31:06.576028 15826 tablet_service.cc:1514] Processing DeleteTablet for tablet 513fad3711e744b48f29af9cf186e126 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:31:06 UTC) from {username='oryx', principal='oryx/127.8.18.254@KRBTEST.COM'} at 127.0.0.1:35360
I20260810 11:31:06.584478 15952 tablet_replica.cc:330] T 513fad3711e744b48f29af9cf186e126 P c19c62abc08e457eb6c66d5baff5682d: stopping tablet replica
I20260810 11:31:06.585273 15952 raft_consensus.cc:2229] T 513fad3711e744b48f29af9cf186e126 P c19c62abc08e457eb6c66d5baff5682d [term 1 LEADER]: Raft consensus shutting down.
I20260810 11:31:06.589320 15953 tablet_replica.cc:330] T 513fad3711e744b48f29af9cf186e126 P b087d7b7ed0f44b585cbb87fa59cda8b: stopping tablet replica
I20260810 11:31:06.590119 15953 raft_consensus.cc:2229] T 513fad3711e744b48f29af9cf186e126 P b087d7b7ed0f44b585cbb87fa59cda8b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 11:31:06.592466 15953 raft_consensus.cc:2258] T 513fad3711e744b48f29af9cf186e126 P b087d7b7ed0f44b585cbb87fa59cda8b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:31:06.591576 15952 raft_consensus.cc:2258] T 513fad3711e744b48f29af9cf186e126 P c19c62abc08e457eb6c66d5baff5682d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:31:06.595474 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 14559
I20260810 11:31:06.596832 14622 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:31:06.603647 15952 ts_tablet_manager.cc:1905] T 513fad3711e744b48f29af9cf186e126 P c19c62abc08e457eb6c66d5baff5682d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:31:06.603968 15954 tablet_replica.cc:330] T 513fad3711e744b48f29af9cf186e126 P 7f6e17ac59cb474d830f9789a7a7f849: stopping tablet replica
I20260810 11:31:06.604652 15954 raft_consensus.cc:2229] T 513fad3711e744b48f29af9cf186e126 P 7f6e17ac59cb474d830f9789a7a7f849 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 11:31:06.605549 15954 raft_consensus.cc:2258] T 513fad3711e744b48f29af9cf186e126 P 7f6e17ac59cb474d830f9789a7a7f849 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:31:06.606801 15953 ts_tablet_manager.cc:1905] T 513fad3711e744b48f29af9cf186e126 P b087d7b7ed0f44b585cbb87fa59cda8b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:31:06.608870 15954 ts_tablet_manager.cc:1905] T 513fad3711e744b48f29af9cf186e126 P 7f6e17ac59cb474d830f9789a7a7f849: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:31:06.621937 15954 ts_tablet_manager.cc:1918] T 513fad3711e744b48f29af9cf186e126 P 7f6e17ac59cb474d830f9789a7a7f849: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 11:31:06.622378 15954 log.cc:1196] T 513fad3711e744b48f29af9cf186e126 P 7f6e17ac59cb474d830f9789a7a7f849: Deleting WAL directory at /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/ts-2/wal/wals/513fad3711e744b48f29af9cf186e126
I20260810 11:31:06.623562 15954 ts_tablet_manager.cc:1939] T 513fad3711e744b48f29af9cf186e126 P 7f6e17ac59cb474d830f9789a7a7f849: Deleting consensus metadata
I20260810 11:31:06.626499 14576 catalog_manager.cc:4841] TS 7f6e17ac59cb474d830f9789a7a7f849 (127.8.18.195:35115): tablet 513fad3711e744b48f29af9cf186e126 (table test-table [id=5a346598604d4e04a539c1922211eb1c]) successfully deleted
I20260810 11:31:06.637938 15953 ts_tablet_manager.cc:1918] T 513fad3711e744b48f29af9cf186e126 P b087d7b7ed0f44b585cbb87fa59cda8b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 11:31:06.638440 15907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:06.580232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35446 (local address 127.8.18.253:44341)
0810 11:31:06.580640 (+ 408us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:06.580650 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:06.596725 (+ 16075us) server_negotiation.cc:407] Connection header received
0810 11:31:06.597164 (+ 439us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:06.597174 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:06.597297 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:06.597395 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:31:06.606179 (+ 8784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:06.607568 (+ 1389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:06.610923 (+ 3355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:06.612232 (+ 1309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:06.612393 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:06.625703 (+ 13310us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:31:06.625711 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:31:06.625717 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:31:06.625779 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:31:06.631769 (+ 5990us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:31:06.633546 (+ 1777us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:31:06.633553 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:31:06.633556 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:31:06.633846 (+ 290us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:31:06.634656 (+ 810us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:31:06.634665 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:31:06.634668 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:31:06.635247 (+ 579us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:31:06.635897 (+ 650us) server_negotiation.cc:1035] Waiting for connection context
0810 11:31:06.637341 (+ 1444us) server_negotiation.cc:299] Negotiation successful
0810 11:31:06.637762 (+ 421us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260810 11:31:06.642024 15952 ts_tablet_manager.cc:1918] T 513fad3711e744b48f29af9cf186e126 P c19c62abc08e457eb6c66d5baff5682d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 11:31:06.642357 15952 log.cc:1196] T 513fad3711e744b48f29af9cf186e126 P c19c62abc08e457eb6c66d5baff5682d: Deleting WAL directory at /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/ts-1/wal/wals/513fad3711e744b48f29af9cf186e126
I20260810 11:31:06.643428 15952 ts_tablet_manager.cc:1939] T 513fad3711e744b48f29af9cf186e126 P c19c62abc08e457eb6c66d5baff5682d: Deleting consensus metadata
I20260810 11:31:06.639238 15953 log.cc:1196] T 513fad3711e744b48f29af9cf186e126 P b087d7b7ed0f44b585cbb87fa59cda8b: Deleting WAL directory at /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/ts-0/wal/wals/513fad3711e744b48f29af9cf186e126
I20260810 11:31:06.649900 15953 ts_tablet_manager.cc:1939] T 513fad3711e744b48f29af9cf186e126 P b087d7b7ed0f44b585cbb87fa59cda8b: Deleting consensus metadata
I20260810 11:31:06.670758 15897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:06.594160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58120 (local address 127.8.18.252:41545)
0810 11:31:06.594358 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:06.594363 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:06.640454 (+ 46091us) server_negotiation.cc:407] Connection header received
0810 11:31:06.640538 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:06.640545 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:06.640654 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:06.640782 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:31:06.644297 (+ 3515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:06.646157 (+ 1860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:06.649513 (+ 3356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:06.650544 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:06.650783 (+ 239us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:06.658261 (+ 7478us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:31:06.658270 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:31:06.658276 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:31:06.658367 (+ 91us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:31:06.665219 (+ 6852us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:31:06.666841 (+ 1622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:31:06.666848 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:31:06.666852 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:31:06.667121 (+ 269us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:31:06.667843 (+ 722us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:31:06.667849 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:31:06.667853 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:31:06.668521 (+ 668us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:31:06.668973 (+ 452us) server_negotiation.cc:1035] Waiting for connection context
0810 11:31:06.669531 (+ 558us) server_negotiation.cc:299] Negotiation successful
0810 11:31:06.669835 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260810 11:31:06.798667 14575 catalog_manager.cc:4841] TS b087d7b7ed0f44b585cbb87fa59cda8b (127.8.18.193:44877): tablet 513fad3711e744b48f29af9cf186e126 (table test-table [id=5a346598604d4e04a539c1922211eb1c]) successfully deleted
I20260810 11:31:06.798874 14577 catalog_manager.cc:4841] TS c19c62abc08e457eb6c66d5baff5682d (127.8.18.194:40169): tablet 513fad3711e744b48f29af9cf186e126 (table test-table [id=5a346598604d4e04a539c1922211eb1c]) successfully deleted
I20260810 11:31:06.799583 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 14559
I20260810 11:31:06.864974 15907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:06.840511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35458 (local address 127.8.18.253:44341)
0810 11:31:06.840773 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:06.840783 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:06.841889 (+ 1106us) server_negotiation.cc:407] Connection header received
0810 11:31:06.841994 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:06.842002 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:06.842107 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:06.842224 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:31:06.844767 (+ 2543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:06.845804 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:06.847987 (+ 2183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:06.848650 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:06.848809 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:06.856482 (+ 7673us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:31:06.856488 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:31:06.856492 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:31:06.856536 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:31:06.861029 (+ 4493us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:31:06.862609 (+ 1580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:31:06.862614 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:31:06.862616 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:31:06.862753 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:31:06.863153 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:31:06.863157 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:31:06.863159 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:31:06.863479 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:31:06.863720 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0810 11:31:06.864290 (+ 570us) server_negotiation.cc:299] Negotiation successful
0810 11:31:06.864562 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260810 11:31:06.866285 15897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:06.842202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58130 (local address 127.8.18.252:41545)
0810 11:31:06.842414 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:06.842420 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:06.842813 (+ 393us) server_negotiation.cc:407] Connection header received
0810 11:31:06.843047 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:06.843054 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:06.843142 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:06.843257 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:31:06.846410 (+ 3153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:06.847184 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:06.848826 (+ 1642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:06.849559 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:06.849693 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:06.856929 (+ 7236us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:31:06.856949 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:31:06.856954 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:31:06.857002 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:31:06.861029 (+ 4027us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:31:06.862309 (+ 1280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:31:06.862314 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:31:06.862317 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:31:06.862467 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:31:06.862939 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:31:06.862947 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:31:06.862950 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:31:06.863375 (+ 425us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:31:06.863645 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0810 11:31:06.865792 (+ 2147us) server_negotiation.cc:299] Negotiation successful
0810 11:31:06.865993 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260810 11:31:07.549259 15965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:07.548064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:36523 (local address 127.8.18.194:34041)
0810 11:31:07.548795 (+ 731us) negotiation.cc:107] Waiting for socket to connect
0810 11:31:07.548986 (+ 191us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.254:36523: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":577,"thread_start_us":412,"threads_started":1}
W20260810 11:31:07.550086 15750 heartbeater.cc:644] Failed to heartbeat to 127.8.18.254:36523 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.8.18.254:36523: connect: Connection refused (error 111)
I20260810 11:31:07.551810 15966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:07.550900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:36523 (local address 127.8.18.193:41923)
0810 11:31:07.551451 (+ 551us) negotiation.cc:107] Waiting for socket to connect
0810 11:31:07.551602 (+ 151us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.254:36523: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":378,"thread_start_us":260,"threads_started":1}
I20260810 11:31:07.552062 15967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:07.550965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:36523 (local address 127.8.18.195:54163)
0810 11:31:07.551696 (+ 731us) negotiation.cc:107] Waiting for socket to connect
0810 11:31:07.551821 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.254:36523: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":608,"thread_start_us":492,"threads_started":1}
W20260810 11:31:07.552471 15592 heartbeater.cc:644] Failed to heartbeat to 127.8.18.254:36523 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.8.18.254:36523: connect: Connection refused (error 111)
W20260810 11:31:07.552477 15894 heartbeater.cc:644] Failed to heartbeat to 127.8.18.254:36523 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.8.18.254:36523: connect: Connection refused (error 111)
I20260810 11:31:08.071954 15968 raft_consensus.cc:482] T 00000000000000000000000000000000 P d1ea41131c2a4c6f984571036be55232 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cd8ee5823e444d28a6cb100898b494c5)
I20260810 11:31:08.072283 15968 raft_consensus.cc:504] T 00000000000000000000000000000000 P d1ea41131c2a4c6f984571036be55232 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "cd8ee5823e444d28a6cb100898b494c5" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 36523 } } peers { permanent_uuid: "8fb82cee99b449c5a33adb02cba79748" member_type: VOTER last_known_addr { host: "127.8.18.253" port: 44341 } attrs { promote: false } } peers { permanent_uuid: "d1ea41131c2a4c6f984571036be55232" member_type: VOTER last_known_addr { host: "127.8.18.252" port: 41545 } attrs { promote: false } }
I20260810 11:31:08.072821 15970 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8fb82cee99b449c5a33adb02cba79748 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cd8ee5823e444d28a6cb100898b494c5)
I20260810 11:31:08.073097 15970 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8fb82cee99b449c5a33adb02cba79748 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "cd8ee5823e444d28a6cb100898b494c5" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 36523 } } peers { permanent_uuid: "8fb82cee99b449c5a33adb02cba79748" member_type: VOTER last_known_addr { host: "127.8.18.253" port: 44341 } attrs { promote: false } } peers { permanent_uuid: "d1ea41131c2a4c6f984571036be55232" member_type: VOTER last_known_addr { host: "127.8.18.252" port: 41545 } attrs { promote: false } }
I20260810 11:31:08.073526 15968 leader_election.cc:288] T 00000000000000000000000000000000 P d1ea41131c2a4c6f984571036be55232 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cd8ee5823e444d28a6cb100898b494c5 (127.8.18.254:36523), 8fb82cee99b449c5a33adb02cba79748 (127.8.18.253:44341)
I20260810 11:31:08.075261 15971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:08.073666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:36523 (local address 127.0.0.1:51406)
0810 11:31:08.074450 (+ 784us) negotiation.cc:107] Waiting for socket to connect
0810 11:31:08.074790 (+ 340us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.254:36523: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":610,"thread_start_us":433,"threads_started":1}
W20260810 11:31:08.075917 15393 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.18.254:36523: connect: Connection refused (error 111)
I20260810 11:31:08.076386 15970 leader_election.cc:288] T 00000000000000000000000000000000 P 8fb82cee99b449c5a33adb02cba79748 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cd8ee5823e444d28a6cb100898b494c5 (127.8.18.254:36523), d1ea41131c2a4c6f984571036be55232 (127.8.18.252:41545)
I20260810 11:31:08.088510 15977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:08.076513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:36523 (local address 127.0.0.1:51416)
0810 11:31:08.087758 (+ 11245us) negotiation.cc:107] Waiting for socket to connect
0810 11:31:08.088056 (+ 298us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.254:36523: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":11040,"threads_started":1}
W20260810 11:31:08.091105 15197 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.18.254:36523: connect: Connection refused (error 111)
I20260810 11:31:08.092692 15971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:08.092180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:36523 (local address 127.0.0.1:51432)
0810 11:31:08.092346 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0810 11:31:08.092469 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.254:36523: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260810 11:31:08.093065 15393 leader_election.cc:334] T 00000000000000000000000000000000 P d1ea41131c2a4c6f984571036be55232 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer cd8ee5823e444d28a6cb100898b494c5 (127.8.18.254:36523): Network error: Client connection negotiation failed: client connection to 127.8.18.254:36523: connect: Connection refused (error 111)
I20260810 11:31:08.094417 15976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:08.093705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:36523 (local address 127.0.0.1:51446)
0810 11:31:08.093969 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0810 11:31:08.094151 (+ 182us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.254:36523: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110}
W20260810 11:31:08.094691 15197 leader_election.cc:334] T 00000000000000000000000000000000 P 8fb82cee99b449c5a33adb02cba79748 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer cd8ee5823e444d28a6cb100898b494c5 (127.8.18.254:36523): Network error: Client connection negotiation failed: client connection to 127.8.18.254:36523: connect: Connection refused (error 111)
I20260810 11:31:08.095651 15974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:08.073967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35460 (local address 127.8.18.253:44341)
0810 11:31:08.082488 (+ 8521us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:08.082499 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:08.082537 (+ 38us) server_negotiation.cc:407] Connection header received
0810 11:31:08.082653 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:08.082662 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:08.082794 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:08.085131 (+ 2337us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:31:08.085489 (+ 358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:08.087150 (+ 1661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:08.093172 (+ 6022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:08.094619 (+ 1447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:08.094802 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:08.094900 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0810 11:31:08.095232 (+ 332us) server_negotiation.cc:299] Negotiation successful
0810 11:31:08.095389 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8317,"thread_start_us":8026,"threads_started":1}
I20260810 11:31:08.095803 15973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:08.073856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.253:44341 (local address 127.0.0.1:35460)
0810 11:31:08.074836 (+ 980us) negotiation.cc:107] Waiting for socket to connect
0810 11:31:08.074868 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0810 11:31:08.075100 (+ 232us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:31:08.084183 (+ 9083us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:31:08.084194 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:31:08.084247 (+ 53us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:31:08.084818 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:08.084837 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:08.089818 (+ 4981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:08.089829 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:08.092945 (+ 3116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:08.092969 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:08.094839 (+ 1870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:08.094848 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:08.095040 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:08.095082 (+ 42us) client_negotiation.cc:769] Sending connection context
0810 11:31:08.095171 (+ 89us) client_negotiation.cc:240] Negotiation successful
0810 11:31:08.095331 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":833,"mutex_wait_us":58,"thread_start_us":630,"threads_started":1}
I20260810 11:31:08.096619 15220 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d1ea41131c2a4c6f984571036be55232" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "8fb82cee99b449c5a33adb02cba79748" is_pre_election: true
I20260810 11:31:08.096985 15220 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 8fb82cee99b449c5a33adb02cba79748 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1ea41131c2a4c6f984571036be55232 in term 1.
I20260810 11:31:08.097561 15391 leader_election.cc:302] T 00000000000000000000000000000000 P d1ea41131c2a4c6f984571036be55232 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fb82cee99b449c5a33adb02cba79748, d1ea41131c2a4c6f984571036be55232; no voters: cd8ee5823e444d28a6cb100898b494c5
I20260810 11:31:08.098013 15977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:08.076609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.252:41545 (local address 127.0.0.1:58134)
0810 11:31:08.089324 (+ 12715us) negotiation.cc:107] Waiting for socket to connect
0810 11:31:08.089349 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0810 11:31:08.089544 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:31:08.090019 (+ 475us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:31:08.090027 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:31:08.090060 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:31:08.090341 (+ 281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:08.090355 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:08.092755 (+ 2400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:08.092762 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:08.095138 (+ 2376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:08.095183 (+ 45us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:08.096909 (+ 1726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:08.096916 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:08.097100 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:08.097151 (+ 51us) client_negotiation.cc:769] Sending connection context
0810 11:31:08.097461 (+ 310us) client_negotiation.cc:240] Negotiation successful
0810 11:31:08.097642 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12585,"mutex_wait_us":74,"thread_start_us":12201,"threads_started":1}
I20260810 11:31:08.097978 15968 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d1ea41131c2a4c6f984571036be55232 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260810 11:31:08.098134 15968 raft_consensus.cc:482] T 00000000000000000000000000000000 P d1ea41131c2a4c6f984571036be55232 [term 1 FOLLOWER]: Starting leader election (detected failure of leader cd8ee5823e444d28a6cb100898b494c5)
I20260810 11:31:08.098201 15978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:08.077071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58134 (local address 127.8.18.252:41545)
0810 11:31:08.089622 (+ 12551us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:08.089629 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:08.089661 (+ 32us) server_negotiation.cc:407] Connection header received
0810 11:31:08.089756 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:08.089762 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:08.089854 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:08.089970 (+ 116us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:31:08.090562 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:08.092517 (+ 1955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:08.095367 (+ 2850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:08.096681 (+ 1314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:08.096838 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:08.096944 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0810 11:31:08.097511 (+ 567us) server_negotiation.cc:299] Negotiation successful
0810 11:31:08.097640 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":12393,"thread_start_us":13570,"threads_started":1}
I20260810 11:31:08.098292 15968 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d1ea41131c2a4c6f984571036be55232 [term 1 FOLLOWER]: Advancing to term 2
I20260810 11:31:08.099392 15416 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8fb82cee99b449c5a33adb02cba79748" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "d1ea41131c2a4c6f984571036be55232" is_pre_election: true
I20260810 11:31:08.100862 15968 raft_consensus.cc:504] T 00000000000000000000000000000000 P d1ea41131c2a4c6f984571036be55232 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "cd8ee5823e444d28a6cb100898b494c5" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 36523 } } peers { permanent_uuid: "8fb82cee99b449c5a33adb02cba79748" member_type: VOTER last_known_addr { host: "127.8.18.253" port: 44341 } attrs { promote: false } } peers { permanent_uuid: "d1ea41131c2a4c6f984571036be55232" member_type: VOTER last_known_addr { host: "127.8.18.252" port: 41545 } attrs { promote: false } }
I20260810 11:31:08.101351 15416 raft_consensus.cc:2379] T 00000000000000000000000000000000 P d1ea41131c2a4c6f984571036be55232 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8fb82cee99b449c5a33adb02cba79748 in current term 2: Already voted for candidate d1ea41131c2a4c6f984571036be55232 in this term.
I20260810 11:31:08.102021 15968 leader_election.cc:288] T 00000000000000000000000000000000 P d1ea41131c2a4c6f984571036be55232 [CANDIDATE]: Term 2 election: Requested vote from peers cd8ee5823e444d28a6cb100898b494c5 (127.8.18.254:36523), 8fb82cee99b449c5a33adb02cba79748 (127.8.18.253:44341)
I20260810 11:31:08.102347 15973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:08.101711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:36523 (local address 127.0.0.1:51460)
0810 11:31:08.101908 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0810 11:31:08.102018 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.254:36523: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260810 11:31:08.102643 15195 leader_election.cc:302] T 00000000000000000000000000000000 P 8fb82cee99b449c5a33adb02cba79748 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8fb82cee99b449c5a33adb02cba79748; no voters: cd8ee5823e444d28a6cb100898b494c5, d1ea41131c2a4c6f984571036be55232
I20260810 11:31:08.102671 15220 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d1ea41131c2a4c6f984571036be55232" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "8fb82cee99b449c5a33adb02cba79748"
I20260810 11:31:08.103016 15220 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8fb82cee99b449c5a33adb02cba79748 [term 1 FOLLOWER]: Advancing to term 2
I20260810 11:31:08.103518 15973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:08.103075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:36523 (local address 127.0.0.1:51464)
0810 11:31:08.103245 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0810 11:31:08.103349 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.254:36523: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260810 11:31:08.103753 15393 leader_election.cc:334] T 00000000000000000000000000000000 P d1ea41131c2a4c6f984571036be55232 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer cd8ee5823e444d28a6cb100898b494c5 (127.8.18.254:36523): Network error: Client connection negotiation failed: client connection to 127.8.18.254:36523: connect: Connection refused (error 111)
I20260810 11:31:08.105551 15220 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 8fb82cee99b449c5a33adb02cba79748 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1ea41131c2a4c6f984571036be55232 in term 2.
I20260810 11:31:08.105762 15970 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 8fb82cee99b449c5a33adb02cba79748 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 11:31:08.106133 15391 leader_election.cc:302] T 00000000000000000000000000000000 P d1ea41131c2a4c6f984571036be55232 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8fb82cee99b449c5a33adb02cba79748, d1ea41131c2a4c6f984571036be55232; no voters: cd8ee5823e444d28a6cb100898b494c5
I20260810 11:31:08.106488 15968 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d1ea41131c2a4c6f984571036be55232 [term 2 FOLLOWER]: Leader election won for term 2
I20260810 11:31:08.107024 15968 raft_consensus.cc:686] T 00000000000000000000000000000000 P d1ea41131c2a4c6f984571036be55232 [term 2 LEADER]: Becoming Leader. State: Replica: d1ea41131c2a4c6f984571036be55232, State: Running, Role: LEADER
I20260810 11:31:08.107458 15968 consensus_queue.cc:237] T 00000000000000000000000000000000 P d1ea41131c2a4c6f984571036be55232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "cd8ee5823e444d28a6cb100898b494c5" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 36523 } } peers { permanent_uuid: "8fb82cee99b449c5a33adb02cba79748" member_type: VOTER last_known_addr { host: "127.8.18.253" port: 44341 } attrs { promote: false } } peers { permanent_uuid: "d1ea41131c2a4c6f984571036be55232" member_type: VOTER last_known_addr { host: "127.8.18.252" port: 41545 } attrs { promote: false } }
I20260810 11:31:08.111629 15980 sys_catalog.cc:455] T 00000000000000000000000000000000 P d1ea41131c2a4c6f984571036be55232 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d1ea41131c2a4c6f984571036be55232. Latest consensus state: current_term: 2 leader_uuid: "d1ea41131c2a4c6f984571036be55232" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "cd8ee5823e444d28a6cb100898b494c5" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 36523 } } peers { permanent_uuid: "8fb82cee99b449c5a33adb02cba79748" member_type: VOTER last_known_addr { host: "127.8.18.253" port: 44341 } attrs { promote: false } } peers { permanent_uuid: "d1ea41131c2a4c6f984571036be55232" member_type: VOTER last_known_addr { host: "127.8.18.252" port: 41545 } attrs { promote: false } } }
I20260810 11:31:08.112092 15980 sys_catalog.cc:458] T 00000000000000000000000000000000 P d1ea41131c2a4c6f984571036be55232 [sys.catalog]: This master's current role is: LEADER
I20260810 11:31:08.112797 15982 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 11:31:08.117307 15982 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 11:31:08.118376 15982 catalog_manager.cc:1259] Loaded cluster ID: 63349ed0186a4282894be80560e2d468
I20260810 11:31:08.118501 15982 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 11:31:08.119695 15982 catalog_manager.cc:1504] Loading token signing keys...
I20260810 11:31:08.120585 15982 catalog_manager.cc:5879] T 00000000000000000000000000000000 P d1ea41131c2a4c6f984571036be55232: Loaded TSK: 0
I20260810 11:31:08.121621 15982 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 11:31:08.167785 15406 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58130:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 11:31:08.175313 15406 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 11:31:08.180011 15973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:08.179464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:36523 (local address 127.0.0.1:51484)
0810 11:31:08.179668 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0810 11:31:08.179840 (+ 172us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.254:36523: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260810 11:31:08.180313 15220 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8fb82cee99b449c5a33adb02cba79748 [term 2 FOLLOWER]: Refusing update from remote peer d1ea41131c2a4c6f984571036be55232: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260810 11:31:08.181366 15973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:08.180813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:36523 (local address 127.0.0.1:51486)
0810 11:31:08.181061 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0810 11:31:08.181161 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.254:36523: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260810 11:31:08.181185 15980 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d1ea41131c2a4c6f984571036be55232 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fb82cee99b449c5a33adb02cba79748" member_type: VOTER last_known_addr { host: "127.8.18.253" port: 44341 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260810 11:31:08.181555 15393 consensus_peers.cc:487] T 00000000000000000000000000000000 P d1ea41131c2a4c6f984571036be55232 -> Peer cd8ee5823e444d28a6cb100898b494c5 (127.8.18.254:36523): Couldn't send request to peer cd8ee5823e444d28a6cb100898b494c5. Status: Network error: Client connection negotiation failed: client connection to 127.8.18.254:36523: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260810 11:31:08.184274 15970 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8fb82cee99b449c5a33adb02cba79748 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d1ea41131c2a4c6f984571036be55232. Latest consensus state: current_term: 2 leader_uuid: "d1ea41131c2a4c6f984571036be55232" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "cd8ee5823e444d28a6cb100898b494c5" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 36523 } } peers { permanent_uuid: "8fb82cee99b449c5a33adb02cba79748" member_type: VOTER last_known_addr { host: "127.8.18.253" port: 44341 } attrs { promote: false } } peers { permanent_uuid: "d1ea41131c2a4c6f984571036be55232" member_type: VOTER last_known_addr { host: "127.8.18.252" port: 41545 } attrs { promote: false } } }
I20260810 11:31:08.184568 15970 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8fb82cee99b449c5a33adb02cba79748 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 11:31:08.186424 15968 sys_catalog.cc:455] T 00000000000000000000000000000000 P d1ea41131c2a4c6f984571036be55232 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "d1ea41131c2a4c6f984571036be55232" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "cd8ee5823e444d28a6cb100898b494c5" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 36523 } } peers { permanent_uuid: "8fb82cee99b449c5a33adb02cba79748" member_type: VOTER last_known_addr { host: "127.8.18.253" port: 44341 } attrs { promote: false } } peers { permanent_uuid: "d1ea41131c2a4c6f984571036be55232" member_type: VOTER last_known_addr { host: "127.8.18.252" port: 41545 } attrs { promote: false } } }
I20260810 11:31:08.186846 15968 sys_catalog.cc:458] T 00000000000000000000000000000000 P d1ea41131c2a4c6f984571036be55232 [sys.catalog]: This master's current role is: LEADER
I20260810 11:31:08.186754 15970 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8fb82cee99b449c5a33adb02cba79748 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "d1ea41131c2a4c6f984571036be55232" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "cd8ee5823e444d28a6cb100898b494c5" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 36523 } } peers { permanent_uuid: "8fb82cee99b449c5a33adb02cba79748" member_type: VOTER last_known_addr { host: "127.8.18.253" port: 44341 } attrs { promote: false } } peers { permanent_uuid: "d1ea41131c2a4c6f984571036be55232" member_type: VOTER last_known_addr { host: "127.8.18.252" port: 41545 } attrs { promote: false } } }
I20260810 11:31:08.187011 15970 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8fb82cee99b449c5a33adb02cba79748 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 11:31:08.209830 15973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:08.196193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.193:44877 (local address 127.0.0.1:44686)
0810 11:31:08.196387 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0810 11:31:08.196407 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 11:31:08.196521 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:31:08.197437 (+ 916us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:31:08.197446 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:31:08.197485 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:31:08.197789 (+ 304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:08.197805 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:08.201143 (+ 3338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:08.201153 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:08.203973 (+ 2820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:08.204026 (+ 53us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:08.208560 (+ 4534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:08.208571 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:08.208920 (+ 349us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:08.208986 (+ 66us) client_negotiation.cc:769] Sending connection context
0810 11:31:08.209099 (+ 113us) client_negotiation.cc:240] Negotiation successful
0810 11:31:08.209349 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":66}
I20260810 11:31:08.213326 15989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:08.196285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44686 (local address 127.8.18.193:44877)
0810 11:31:08.197011 (+ 726us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:08.197018 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:08.197037 (+ 19us) server_negotiation.cc:407] Connection header received
0810 11:31:08.197111 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:08.197116 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:08.197191 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:08.197986 (+ 795us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:31:08.198261 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:08.200878 (+ 2617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:08.206375 (+ 5497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:08.208267 (+ 1892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:08.212575 (+ 4308us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:08.212723 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0810 11:31:08.212783 (+ 60us) server_negotiation.cc:299] Negotiation successful
0810 11:31:08.212968 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":586,"thread_start_us":463,"threads_started":1}
I20260810 11:31:08.214392 15524 tablet_service.cc:1467] Processing CreateTablet for tablet e4039ed6375d4bb698b8f762ab397ca3 (DEFAULT_TABLE table=test-table [id=b0acdaaf9511493bafb77d6dbb2deeee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:31:08.214955 15524 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e4039ed6375d4bb698b8f762ab397ca3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:31:08.221665 15973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:08.197092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.195:35115 (local address 127.0.0.1:35362)
0810 11:31:08.210154 (+ 13062us) negotiation.cc:107] Waiting for socket to connect
0810 11:31:08.210176 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 11:31:08.210331 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:31:08.214975 (+ 4644us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:31:08.214979 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:31:08.215037 (+ 58us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:31:08.215230 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:08.215240 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:08.217050 (+ 1810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:08.217057 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:08.219068 (+ 2011us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:08.219088 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:08.220830 (+ 1742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:08.220835 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:08.221004 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:08.221031 (+ 27us) client_negotiation.cc:769] Sending connection context
0810 11:31:08.221165 (+ 134us) client_negotiation.cc:240] Negotiation successful
0810 11:31:08.221279 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12958,"thread_start_us":16957,"threads_started":1}
I20260810 11:31:08.221776 15992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:08.197567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35362 (local address 127.8.18.195:35115)
0810 11:31:08.214605 (+ 17038us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:08.214612 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:08.214632 (+ 20us) server_negotiation.cc:407] Connection header received
0810 11:31:08.214722 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:08.214727 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:08.214824 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:08.214943 (+ 119us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:31:08.215414 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:08.216817 (+ 1403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:08.219356 (+ 2539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:08.220601 (+ 1245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:08.220765 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:08.220881 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0810 11:31:08.221228 (+ 347us) server_negotiation.cc:299] Negotiation successful
0810 11:31:08.221399 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16894,"thread_start_us":16771,"threads_started":1}
I20260810 11:31:08.222760 15826 tablet_service.cc:1467] Processing CreateTablet for tablet e4039ed6375d4bb698b8f762ab397ca3 (DEFAULT_TABLE table=test-table [id=b0acdaaf9511493bafb77d6dbb2deeee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:31:08.222990 15993 tablet_bootstrap.cc:492] T e4039ed6375d4bb698b8f762ab397ca3 P b087d7b7ed0f44b585cbb87fa59cda8b: Bootstrap starting.
I20260810 11:31:08.223278 15826 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e4039ed6375d4bb698b8f762ab397ca3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:31:08.224973 15993 tablet_bootstrap.cc:654] T e4039ed6375d4bb698b8f762ab397ca3 P b087d7b7ed0f44b585cbb87fa59cda8b: Neither blocks nor log segments found. Creating new log.
I20260810 11:31:08.227171 15993 tablet_bootstrap.cc:492] T e4039ed6375d4bb698b8f762ab397ca3 P b087d7b7ed0f44b585cbb87fa59cda8b: No bootstrap required, opened a new log
I20260810 11:31:08.227357 15993 ts_tablet_manager.cc:1397] T e4039ed6375d4bb698b8f762ab397ca3 P b087d7b7ed0f44b585cbb87fa59cda8b: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20260810 11:31:08.228070 15993 raft_consensus.cc:348] T e4039ed6375d4bb698b8f762ab397ca3 P b087d7b7ed0f44b585cbb87fa59cda8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b087d7b7ed0f44b585cbb87fa59cda8b" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 44877 } } peers { permanent_uuid: "c19c62abc08e457eb6c66d5baff5682d" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 40169 } } peers { permanent_uuid: "7f6e17ac59cb474d830f9789a7a7f849" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 35115 } }
I20260810 11:31:08.228338 15993 raft_consensus.cc:374] T e4039ed6375d4bb698b8f762ab397ca3 P b087d7b7ed0f44b585cbb87fa59cda8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:31:08.228446 15993 raft_consensus.cc:729] T e4039ed6375d4bb698b8f762ab397ca3 P b087d7b7ed0f44b585cbb87fa59cda8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b087d7b7ed0f44b585cbb87fa59cda8b, State: Initialized, Role: FOLLOWER
I20260810 11:31:08.228770 15993 consensus_queue.cc:260] T e4039ed6375d4bb698b8f762ab397ca3 P b087d7b7ed0f44b585cbb87fa59cda8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b087d7b7ed0f44b585cbb87fa59cda8b" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 44877 } } peers { permanent_uuid: "c19c62abc08e457eb6c66d5baff5682d" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 40169 } } peers { permanent_uuid: "7f6e17ac59cb474d830f9789a7a7f849" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 35115 } }
I20260810 11:31:08.229202 15971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:08.196636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.194:40169 (local address 127.0.0.1:47042)
0810 11:31:08.197112 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0810 11:31:08.197137 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0810 11:31:08.197258 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:31:08.213121 (+ 15863us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:31:08.213127 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:31:08.213171 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:31:08.213555 (+ 384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:08.213571 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:08.224168 (+ 10597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:08.224176 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:08.226269 (+ 2093us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:08.226287 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:08.228228 (+ 1941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:08.228235 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:08.228532 (+ 297us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:08.228582 (+ 50us) client_negotiation.cc:769] Sending connection context
0810 11:31:08.228698 (+ 116us) client_negotiation.cc:240] Negotiation successful
0810 11:31:08.228875 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":366}
I20260810 11:31:08.229526 15990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:08.197109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47042 (local address 127.8.18.194:40169)
0810 11:31:08.212662 (+ 15553us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:08.212668 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:08.212689 (+ 21us) server_negotiation.cc:407] Connection header received
0810 11:31:08.212770 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:08.212774 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:08.212852 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:08.221595 (+ 8743us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:31:08.222246 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:08.223960 (+ 1714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:08.226664 (+ 2704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:08.227948 (+ 1284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:08.228110 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:08.228252 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0810 11:31:08.228758 (+ 506us) server_negotiation.cc:299] Negotiation successful
0810 11:31:08.228926 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15411,"thread_start_us":15833,"threads_started":1}
I20260810 11:31:08.230516 15681 tablet_service.cc:1467] Processing CreateTablet for tablet e4039ed6375d4bb698b8f762ab397ca3 (DEFAULT_TABLE table=test-table [id=b0acdaaf9511493bafb77d6dbb2deeee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:31:08.230953 15681 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e4039ed6375d4bb698b8f762ab397ca3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:31:08.237664 15998 tablet_bootstrap.cc:492] T e4039ed6375d4bb698b8f762ab397ca3 P c19c62abc08e457eb6c66d5baff5682d: Bootstrap starting.
I20260810 11:31:08.237862 15993 ts_tablet_manager.cc:1428] T e4039ed6375d4bb698b8f762ab397ca3 P b087d7b7ed0f44b585cbb87fa59cda8b: Time spent starting tablet: real 0.010s user 0.002s sys 0.002s
I20260810 11:31:08.238168 15591 heartbeater.cc:503] Master 127.8.18.252:41545 was elected leader, sending a full tablet report...
I20260810 11:31:08.241466 15998 tablet_bootstrap.cc:654] T e4039ed6375d4bb698b8f762ab397ca3 P c19c62abc08e457eb6c66d5baff5682d: Neither blocks nor log segments found. Creating new log.
I20260810 11:31:08.245568 15997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:08.231534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:36523 (local address 127.8.18.193:49167)
0810 11:31:08.244873 (+ 13339us) negotiation.cc:107] Waiting for socket to connect
0810 11:31:08.245108 (+ 235us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.254:36523: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":13124,"threads_started":1}
I20260810 11:31:08.247277 15996 tablet_bootstrap.cc:492] T e4039ed6375d4bb698b8f762ab397ca3 P 7f6e17ac59cb474d830f9789a7a7f849: Bootstrap starting.
I20260810 11:31:08.247471 15997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:08.246870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:36523 (local address 127.8.18.193:33121)
0810 11:31:08.247127 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0810 11:31:08.247259 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.254:36523: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260810 11:31:08.249243 15996 tablet_bootstrap.cc:654] T e4039ed6375d4bb698b8f762ab397ca3 P 7f6e17ac59cb474d830f9789a7a7f849: Neither blocks nor log segments found. Creating new log.
I20260810 11:31:08.250975 15998 tablet_bootstrap.cc:492] T e4039ed6375d4bb698b8f762ab397ca3 P c19c62abc08e457eb6c66d5baff5682d: No bootstrap required, opened a new log
I20260810 11:31:08.251163 15998 ts_tablet_manager.cc:1397] T e4039ed6375d4bb698b8f762ab397ca3 P c19c62abc08e457eb6c66d5baff5682d: Time spent bootstrapping tablet: real 0.014s user 0.002s sys 0.008s
I20260810 11:31:08.251951 15996 tablet_bootstrap.cc:492] T e4039ed6375d4bb698b8f762ab397ca3 P 7f6e17ac59cb474d830f9789a7a7f849: No bootstrap required, opened a new log
I20260810 11:31:08.252146 15996 ts_tablet_manager.cc:1397] T e4039ed6375d4bb698b8f762ab397ca3 P 7f6e17ac59cb474d830f9789a7a7f849: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260810 11:31:08.252127 15998 raft_consensus.cc:348] T e4039ed6375d4bb698b8f762ab397ca3 P c19c62abc08e457eb6c66d5baff5682d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b087d7b7ed0f44b585cbb87fa59cda8b" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 44877 } } peers { permanent_uuid: "c19c62abc08e457eb6c66d5baff5682d" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 40169 } } peers { permanent_uuid: "7f6e17ac59cb474d830f9789a7a7f849" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 35115 } }
I20260810 11:31:08.252388 15998 raft_consensus.cc:374] T e4039ed6375d4bb698b8f762ab397ca3 P c19c62abc08e457eb6c66d5baff5682d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:31:08.252506 15998 raft_consensus.cc:729] T e4039ed6375d4bb698b8f762ab397ca3 P c19c62abc08e457eb6c66d5baff5682d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c19c62abc08e457eb6c66d5baff5682d, State: Initialized, Role: FOLLOWER
I20260810 11:31:08.252779 15998 consensus_queue.cc:260] T e4039ed6375d4bb698b8f762ab397ca3 P c19c62abc08e457eb6c66d5baff5682d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b087d7b7ed0f44b585cbb87fa59cda8b" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 44877 } } peers { permanent_uuid: "c19c62abc08e457eb6c66d5baff5682d" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 40169 } } peers { permanent_uuid: "7f6e17ac59cb474d830f9789a7a7f849" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 35115 } }
I20260810 11:31:08.253093 15996 raft_consensus.cc:348] T e4039ed6375d4bb698b8f762ab397ca3 P 7f6e17ac59cb474d830f9789a7a7f849 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b087d7b7ed0f44b585cbb87fa59cda8b" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 44877 } } peers { permanent_uuid: "c19c62abc08e457eb6c66d5baff5682d" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 40169 } } peers { permanent_uuid: "7f6e17ac59cb474d830f9789a7a7f849" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 35115 } }
I20260810 11:31:08.253438 15996 raft_consensus.cc:374] T e4039ed6375d4bb698b8f762ab397ca3 P 7f6e17ac59cb474d830f9789a7a7f849 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:31:08.253566 15996 raft_consensus.cc:729] T e4039ed6375d4bb698b8f762ab397ca3 P 7f6e17ac59cb474d830f9789a7a7f849 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f6e17ac59cb474d830f9789a7a7f849, State: Initialized, Role: FOLLOWER
I20260810 11:31:08.254974 15998 ts_tablet_manager.cc:1428] T e4039ed6375d4bb698b8f762ab397ca3 P c19c62abc08e457eb6c66d5baff5682d: Time spent starting tablet: real 0.004s user 0.001s sys 0.002s
I20260810 11:31:08.256072 16002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:08.254229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:36523 (local address 127.8.18.194:36167)
0810 11:31:08.255642 (+ 1413us) negotiation.cc:107] Waiting for socket to connect
0810 11:31:08.255797 (+ 155us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.254:36523: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1272,"thread_start_us":1126,"threads_started":1}
I20260810 11:31:08.256873 15749 heartbeater.cc:503] Master 127.8.18.252:41545 was elected leader, sending a full tablet report...
I20260810 11:31:08.256659 15996 consensus_queue.cc:260] T e4039ed6375d4bb698b8f762ab397ca3 P 7f6e17ac59cb474d830f9789a7a7f849 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b087d7b7ed0f44b585cbb87fa59cda8b" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 44877 } } peers { permanent_uuid: "c19c62abc08e457eb6c66d5baff5682d" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 40169 } } peers { permanent_uuid: "7f6e17ac59cb474d830f9789a7a7f849" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 35115 } }
I20260810 11:31:08.259651 15996 ts_tablet_manager.cc:1428] T e4039ed6375d4bb698b8f762ab397ca3 P 7f6e17ac59cb474d830f9789a7a7f849: Time spent starting tablet: real 0.007s user 0.002s sys 0.003s
I20260810 11:31:08.261579 16002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:08.256771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:36523 (local address 127.8.18.194:40113)
0810 11:31:08.261126 (+ 4355us) negotiation.cc:107] Waiting for socket to connect
0810 11:31:08.261282 (+ 156us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.254:36523: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":4183,"threads_started":1}
I20260810 11:31:08.262338 15893 heartbeater.cc:503] Master 127.8.18.252:41545 was elected leader, sending a full tablet report...
I20260810 11:31:08.266819 16005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:08.260447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:36523 (local address 127.8.18.195:49861)
0810 11:31:08.266461 (+ 6014us) negotiation.cc:107] Waiting for socket to connect
0810 11:31:08.266590 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.254:36523: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":5872,"thread_start_us":5620,"threads_started":1}
I20260810 11:31:08.268041 16005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:08.267638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:36523 (local address 127.8.18.195:50715)
0810 11:31:08.267843 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0810 11:31:08.267920 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.254:36523: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260810 11:31:08.270677 16001 raft_consensus.cc:482] T e4039ed6375d4bb698b8f762ab397ca3 P c19c62abc08e457eb6c66d5baff5682d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 11:31:08.270936 16001 raft_consensus.cc:504] T e4039ed6375d4bb698b8f762ab397ca3 P c19c62abc08e457eb6c66d5baff5682d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b087d7b7ed0f44b585cbb87fa59cda8b" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 44877 } } peers { permanent_uuid: "c19c62abc08e457eb6c66d5baff5682d" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 40169 } } peers { permanent_uuid: "7f6e17ac59cb474d830f9789a7a7f849" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 35115 } }
I20260810 11:31:08.271826 16001 leader_election.cc:288] T e4039ed6375d4bb698b8f762ab397ca3 P c19c62abc08e457eb6c66d5baff5682d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b087d7b7ed0f44b585cbb87fa59cda8b (127.8.18.193:44877), 7f6e17ac59cb474d830f9789a7a7f849 (127.8.18.195:35115)
I20260810 11:31:08.272513 15848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4039ed6375d4bb698b8f762ab397ca3" candidate_uuid: "c19c62abc08e457eb6c66d5baff5682d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f6e17ac59cb474d830f9789a7a7f849" is_pre_election: true
I20260810 11:31:08.272536 15546 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4039ed6375d4bb698b8f762ab397ca3" candidate_uuid: "c19c62abc08e457eb6c66d5baff5682d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b087d7b7ed0f44b585cbb87fa59cda8b" is_pre_election: true
I20260810 11:31:08.272792 15848 raft_consensus.cc:2454] T e4039ed6375d4bb698b8f762ab397ca3 P 7f6e17ac59cb474d830f9789a7a7f849 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c19c62abc08e457eb6c66d5baff5682d in term 0.
I20260810 11:31:08.272912 15546 raft_consensus.cc:2454] T e4039ed6375d4bb698b8f762ab397ca3 P b087d7b7ed0f44b585cbb87fa59cda8b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c19c62abc08e457eb6c66d5baff5682d in term 0.
I20260810 11:31:08.273470 15633 leader_election.cc:302] T e4039ed6375d4bb698b8f762ab397ca3 P c19c62abc08e457eb6c66d5baff5682d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7f6e17ac59cb474d830f9789a7a7f849, c19c62abc08e457eb6c66d5baff5682d; no voters:
I20260810 11:31:08.273816 16001 raft_consensus.cc:2781] T e4039ed6375d4bb698b8f762ab397ca3 P c19c62abc08e457eb6c66d5baff5682d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 11:31:08.273967 16001 raft_consensus.cc:482] T e4039ed6375d4bb698b8f762ab397ca3 P c19c62abc08e457eb6c66d5baff5682d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 11:31:08.274078 16001 raft_consensus.cc:3037] T e4039ed6375d4bb698b8f762ab397ca3 P c19c62abc08e457eb6c66d5baff5682d [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:31:08.276553 16001 raft_consensus.cc:504] T e4039ed6375d4bb698b8f762ab397ca3 P c19c62abc08e457eb6c66d5baff5682d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b087d7b7ed0f44b585cbb87fa59cda8b" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 44877 } } peers { permanent_uuid: "c19c62abc08e457eb6c66d5baff5682d" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 40169 } } peers { permanent_uuid: "7f6e17ac59cb474d830f9789a7a7f849" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 35115 } }
I20260810 11:31:08.277257 16001 leader_election.cc:288] T e4039ed6375d4bb698b8f762ab397ca3 P c19c62abc08e457eb6c66d5baff5682d [CANDIDATE]: Term 1 election: Requested vote from peers b087d7b7ed0f44b585cbb87fa59cda8b (127.8.18.193:44877), 7f6e17ac59cb474d830f9789a7a7f849 (127.8.18.195:35115)
I20260810 11:31:08.278055 15546 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4039ed6375d4bb698b8f762ab397ca3" candidate_uuid: "c19c62abc08e457eb6c66d5baff5682d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b087d7b7ed0f44b585cbb87fa59cda8b"
I20260810 11:31:08.278055 15848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4039ed6375d4bb698b8f762ab397ca3" candidate_uuid: "c19c62abc08e457eb6c66d5baff5682d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f6e17ac59cb474d830f9789a7a7f849"
I20260810 11:31:08.278349 15848 raft_consensus.cc:3037] T e4039ed6375d4bb698b8f762ab397ca3 P 7f6e17ac59cb474d830f9789a7a7f849 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:31:08.278343 15546 raft_consensus.cc:3037] T e4039ed6375d4bb698b8f762ab397ca3 P b087d7b7ed0f44b585cbb87fa59cda8b [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:31:08.281913 15546 raft_consensus.cc:2454] T e4039ed6375d4bb698b8f762ab397ca3 P b087d7b7ed0f44b585cbb87fa59cda8b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c19c62abc08e457eb6c66d5baff5682d in term 1.
I20260810 11:31:08.281913 15848 raft_consensus.cc:2454] T e4039ed6375d4bb698b8f762ab397ca3 P 7f6e17ac59cb474d830f9789a7a7f849 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c19c62abc08e457eb6c66d5baff5682d in term 1.
I20260810 11:31:08.282655 15632 leader_election.cc:302] T e4039ed6375d4bb698b8f762ab397ca3 P c19c62abc08e457eb6c66d5baff5682d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 7f6e17ac59cb474d830f9789a7a7f849, b087d7b7ed0f44b585cbb87fa59cda8b, c19c62abc08e457eb6c66d5baff5682d; no voters:
I20260810 11:31:08.283104 16001 raft_consensus.cc:2781] T e4039ed6375d4bb698b8f762ab397ca3 P c19c62abc08e457eb6c66d5baff5682d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:31:08.283730 16001 raft_consensus.cc:686] T e4039ed6375d4bb698b8f762ab397ca3 P c19c62abc08e457eb6c66d5baff5682d [term 1 LEADER]: Becoming Leader. State: Replica: c19c62abc08e457eb6c66d5baff5682d, State: Running, Role: LEADER
I20260810 11:31:08.284099 16001 consensus_queue.cc:237] T e4039ed6375d4bb698b8f762ab397ca3 P c19c62abc08e457eb6c66d5baff5682d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b087d7b7ed0f44b585cbb87fa59cda8b" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 44877 } } peers { permanent_uuid: "c19c62abc08e457eb6c66d5baff5682d" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 40169 } } peers { permanent_uuid: "7f6e17ac59cb474d830f9789a7a7f849" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 35115 } }
I20260810 11:31:08.285576 16003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:08.284769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:36523 (local address 127.8.18.194:43583)
0810 11:31:08.284974 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0810 11:31:08.285116 (+ 142us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.254:36523: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260810 11:31:08.287209 15406 catalog_manager.cc:5495] T e4039ed6375d4bb698b8f762ab397ca3 P c19c62abc08e457eb6c66d5baff5682d reported cstate change: term changed from 0 to 1, leader changed from <none> to c19c62abc08e457eb6c66d5baff5682d (127.8.18.194). New cstate: current_term: 1 leader_uuid: "c19c62abc08e457eb6c66d5baff5682d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b087d7b7ed0f44b585cbb87fa59cda8b" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 44877 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c19c62abc08e457eb6c66d5baff5682d" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 40169 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7f6e17ac59cb474d830f9789a7a7f849" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 35115 } health_report { overall_health: UNKNOWN } } }
I20260810 11:31:08.317054 15990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:08.311862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47052 (local address 127.8.18.194:40169)
0810 11:31:08.312080 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:08.312085 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:08.312104 (+ 19us) server_negotiation.cc:407] Connection header received
0810 11:31:08.312285 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:08.312289 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:08.312371 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:08.312451 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 11:31:08.312899 (+ 448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:08.313826 (+ 927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:08.315230 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:08.315682 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:08.315787 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:08.316284 (+ 497us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 11:31:08.316398 (+ 114us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 11:31:08.316474 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0810 11:31:08.316620 (+ 146us) server_negotiation.cc:299] Negotiation successful
0810 11:31:08.316695 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260810 11:31:08.320561 15848 raft_consensus.cc:1261] T e4039ed6375d4bb698b8f762ab397ca3 P 7f6e17ac59cb474d830f9789a7a7f849 [term 1 FOLLOWER]: Refusing update from remote peer c19c62abc08e457eb6c66d5baff5682d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 11:31:08.320613 15546 raft_consensus.cc:1261] T e4039ed6375d4bb698b8f762ab397ca3 P b087d7b7ed0f44b585cbb87fa59cda8b [term 1 FOLLOWER]: Refusing update from remote peer c19c62abc08e457eb6c66d5baff5682d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 11:31:08.321357 16001 consensus_queue.cc:1035] T e4039ed6375d4bb698b8f762ab397ca3 P c19c62abc08e457eb6c66d5baff5682d [LEADER]: Connected to new peer: Peer: permanent_uuid: "7f6e17ac59cb474d830f9789a7a7f849" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 11:31:08.321719 16006 consensus_queue.cc:1035] T e4039ed6375d4bb698b8f762ab397ca3 P c19c62abc08e457eb6c66d5baff5682d [LEADER]: Connected to new peer: Peer: permanent_uuid: "b087d7b7ed0f44b585cbb87fa59cda8b" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 44877 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 11:31:08.325510 15997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:08.324330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:36523 (local address 127.8.18.193:40825)
0810 11:31:08.325145 (+ 815us) negotiation.cc:107] Waiting for socket to connect
0810 11:31:08.325235 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.254:36523: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":702}
I20260810 11:31:08.325857 16005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:08.325291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:36523 (local address 127.8.18.195:59313)
0810 11:31:08.325542 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0810 11:31:08.325676 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.254:36523: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260810 11:31:08.334092 16005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:08.333687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:36523 (local address 127.8.18.195:35441)
0810 11:31:08.333823 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0810 11:31:08.333918 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.254:36523: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260810 11:31:08.338155 15997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:08.337624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:36523 (local address 127.8.18.193:43581)
0810 11:31:08.337880 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0810 11:31:08.337982 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.254:36523: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138}
I20260810 11:31:08.340286 16003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:08.339523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:36523 (local address 127.8.18.194:36145)
0810 11:31:08.339904 (+ 381us) negotiation.cc:107] Waiting for socket to connect
0810 11:31:08.340027 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.254:36523: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":228}
I20260810 11:31:08.340826 16010 mvcc.cc:205] Tried to move back new op lower bound from 7316936574234411008 to 7316936574094311424. Current Snapshot: MvccSnapshot[applied={T|T < 7316936574234411008}]
I20260810 11:31:08.344874 15405 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58130:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 11:31:08.345271 15405 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58130:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 11:31:08.352701 15405 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d1ea41131c2a4c6f984571036be55232: Sending DeleteTablet for 3 replicas of tablet e4039ed6375d4bb698b8f762ab397ca3
I20260810 11:31:08.353919 15826 tablet_service.cc:1514] Processing DeleteTablet for tablet e4039ed6375d4bb698b8f762ab397ca3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:31:08 UTC) from {username='oryx', principal='oryx/127.8.18.252@KRBTEST.COM'} at 127.0.0.1:35362
I20260810 11:31:08.353987 15524 tablet_service.cc:1514] Processing DeleteTablet for tablet e4039ed6375d4bb698b8f762ab397ca3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:31:08 UTC) from {username='oryx', principal='oryx/127.8.18.252@KRBTEST.COM'} at 127.0.0.1:44686
I20260810 11:31:08.353987 15681 tablet_service.cc:1514] Processing DeleteTablet for tablet e4039ed6375d4bb698b8f762ab397ca3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:31:08 UTC) from {username='oryx', principal='oryx/127.8.18.252@KRBTEST.COM'} at 127.0.0.1:47042
I20260810 11:31:08.354584 15680 tablet_service.cc:1514] Processing DeleteTablet for tablet e4039ed6375d4bb698b8f762ab397ca3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:31:08 UTC) from {username='oryx', principal='oryx/127.8.18.252@KRBTEST.COM'} at 127.0.0.1:47042
I20260810 11:31:08.356402 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 15370
I20260810 11:31:08.362026 16018 tablet_replica.cc:330] T e4039ed6375d4bb698b8f762ab397ca3 P 7f6e17ac59cb474d830f9789a7a7f849: stopping tablet replica
I20260810 11:31:08.362318 16018 raft_consensus.cc:2229] T e4039ed6375d4bb698b8f762ab397ca3 P 7f6e17ac59cb474d830f9789a7a7f849 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 11:31:08.362661 16018 raft_consensus.cc:2258] T e4039ed6375d4bb698b8f762ab397ca3 P 7f6e17ac59cb474d830f9789a7a7f849 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:31:08.363374 16019 tablet_replica.cc:330] T e4039ed6375d4bb698b8f762ab397ca3 P b087d7b7ed0f44b585cbb87fa59cda8b: stopping tablet replica
I20260810 11:31:08.363601 16019 raft_consensus.cc:2229] T e4039ed6375d4bb698b8f762ab397ca3 P b087d7b7ed0f44b585cbb87fa59cda8b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 11:31:08.364017 16019 raft_consensus.cc:2258] T e4039ed6375d4bb698b8f762ab397ca3 P b087d7b7ed0f44b585cbb87fa59cda8b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:31:08.365211 16018 ts_tablet_manager.cc:1905] T e4039ed6375d4bb698b8f762ab397ca3 P 7f6e17ac59cb474d830f9789a7a7f849: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:31:08.365644 16019 ts_tablet_manager.cc:1905] T e4039ed6375d4bb698b8f762ab397ca3 P b087d7b7ed0f44b585cbb87fa59cda8b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:31:08.366703 16020 tablet_replica.cc:330] T e4039ed6375d4bb698b8f762ab397ca3 P c19c62abc08e457eb6c66d5baff5682d: stopping tablet replica
I20260810 11:31:08.366937 16020 raft_consensus.cc:2229] T e4039ed6375d4bb698b8f762ab397ca3 P c19c62abc08e457eb6c66d5baff5682d [term 1 LEADER]: Raft consensus shutting down.
I20260810 11:31:08.367468 16020 raft_consensus.cc:2258] T e4039ed6375d4bb698b8f762ab397ca3 P c19c62abc08e457eb6c66d5baff5682d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:31:08.370651 16020 ts_tablet_manager.cc:1905] T e4039ed6375d4bb698b8f762ab397ca3 P c19c62abc08e457eb6c66d5baff5682d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:31:08.372026 16019 ts_tablet_manager.cc:1918] T e4039ed6375d4bb698b8f762ab397ca3 P b087d7b7ed0f44b585cbb87fa59cda8b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 11:31:08.372205 16019 log.cc:1196] T e4039ed6375d4bb698b8f762ab397ca3 P b087d7b7ed0f44b585cbb87fa59cda8b: Deleting WAL directory at /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/ts-0/wal/wals/e4039ed6375d4bb698b8f762ab397ca3
I20260810 11:31:08.372778 16019 ts_tablet_manager.cc:1939] T e4039ed6375d4bb698b8f762ab397ca3 P b087d7b7ed0f44b585cbb87fa59cda8b: Deleting consensus metadata
I20260810 11:31:08.374300 15389 catalog_manager.cc:4841] TS b087d7b7ed0f44b585cbb87fa59cda8b (127.8.18.193:44877): tablet e4039ed6375d4bb698b8f762ab397ca3 (table test-table [id=b0acdaaf9511493bafb77d6dbb2deeee]) successfully deleted
I20260810 11:31:08.376443 16018 ts_tablet_manager.cc:1918] T e4039ed6375d4bb698b8f762ab397ca3 P 7f6e17ac59cb474d830f9789a7a7f849: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 11:31:08.376627 16018 log.cc:1196] T e4039ed6375d4bb698b8f762ab397ca3 P 7f6e17ac59cb474d830f9789a7a7f849: Deleting WAL directory at /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/ts-2/wal/wals/e4039ed6375d4bb698b8f762ab397ca3
I20260810 11:31:08.377091 16018 ts_tablet_manager.cc:1939] T e4039ed6375d4bb698b8f762ab397ca3 P 7f6e17ac59cb474d830f9789a7a7f849: Deleting consensus metadata
I20260810 11:31:08.377992 16020 ts_tablet_manager.cc:1918] T e4039ed6375d4bb698b8f762ab397ca3 P c19c62abc08e457eb6c66d5baff5682d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 11:31:08.378178 16020 log.cc:1196] T e4039ed6375d4bb698b8f762ab397ca3 P c19c62abc08e457eb6c66d5baff5682d: Deleting WAL directory at /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786361401922362-8267-0/minicluster-data/ts-1/wal/wals/e4039ed6375d4bb698b8f762ab397ca3
I20260810 11:31:08.378741 15391 catalog_manager.cc:4841] TS 7f6e17ac59cb474d830f9789a7a7f849 (127.8.18.195:35115): tablet e4039ed6375d4bb698b8f762ab397ca3 (table test-table [id=b0acdaaf9511493bafb77d6dbb2deeee]) successfully deleted
I20260810 11:31:08.378906 16020 ts_tablet_manager.cc:1939] T e4039ed6375d4bb698b8f762ab397ca3 P c19c62abc08e457eb6c66d5baff5682d: Deleting consensus metadata
I20260810 11:31:08.380183 15392 catalog_manager.cc:4841] TS c19c62abc08e457eb6c66d5baff5682d (127.8.18.194:40169): tablet e4039ed6375d4bb698b8f762ab397ca3 (table test-table [id=b0acdaaf9511493bafb77d6dbb2deeee]) successfully deleted
W20260810 11:31:08.380793 15392 catalog_manager.cc:4808] TS c19c62abc08e457eb6c66d5baff5682d (127.8.18.194:40169): delete failed for tablet e4039ed6375d4bb698b8f762ab397ca3 because the tablet was not found. No further retry: Not found: Tablet not found: e4039ed6375d4bb698b8f762ab397ca3
I20260810 11:31:08.388060 15989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:08.358028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44702 (local address 127.8.18.193:44877)
0810 11:31:08.358354 (+ 326us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:08.358364 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:08.366009 (+ 7645us) server_negotiation.cc:407] Connection header received
0810 11:31:08.366251 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:08.366257 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:08.366367 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:08.366503 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:31:08.369981 (+ 3478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:08.371171 (+ 1190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:08.373110 (+ 1939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:08.373841 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:08.373984 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:08.381117 (+ 7133us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:31:08.381137 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:31:08.381151 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:31:08.381195 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:31:08.384561 (+ 3366us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:31:08.385571 (+ 1010us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:31:08.385581 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:31:08.385583 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:31:08.385765 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:31:08.386247 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:31:08.386251 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:31:08.386253 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:31:08.386694 (+ 441us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:31:08.387032 (+ 338us) server_negotiation.cc:1035] Waiting for connection context
0810 11:31:08.387429 (+ 397us) server_negotiation.cc:299] Negotiation successful
0810 11:31:08.387719 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260810 11:31:08.388799 15586 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:31:08.488571 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 15370
I20260810 11:31:08.508857 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 15604
I20260810 11:31:08.524125 15990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:08.509760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47064 (local address 127.8.18.194:40169)
0810 11:31:08.510009 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:08.510018 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:08.510128 (+ 110us) server_negotiation.cc:407] Connection header received
0810 11:31:08.510361 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:08.510366 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:08.510444 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:08.510561 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:31:08.511940 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:08.512704 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:08.514261 (+ 1557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:08.514701 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:08.514846 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:08.517751 (+ 2905us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:31:08.517770 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:31:08.517787 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:31:08.517831 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:31:08.520918 (+ 3087us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:31:08.521835 (+ 917us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:31:08.521843 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:31:08.521846 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:31:08.522002 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:31:08.522429 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:31:08.522433 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:31:08.522435 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:31:08.522832 (+ 397us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:31:08.523202 (+ 370us) server_negotiation.cc:1035] Waiting for connection context
0810 11:31:08.523484 (+ 282us) server_negotiation.cc:299] Negotiation successful
0810 11:31:08.523738 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260810 11:31:08.524914 15744 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:31:08.591027 15971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:08.590426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:36523 (local address 127.0.0.1:51488)
0810 11:31:08.590675 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0810 11:31:08.590822 (+ 147us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.254:36523: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
I20260810 11:31:08.592473 15971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:08.591799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:36523 (local address 127.0.0.1:51496)
0810 11:31:08.592180 (+ 381us) negotiation.cc:107] Waiting for socket to connect
0810 11:31:08.592281 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.254:36523: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194}
I20260810 11:31:08.652840 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 15604
I20260810 11:31:08.679090 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 15762
I20260810 11:31:08.694797 15992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:08.679996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35364 (local address 127.8.18.195:35115)
0810 11:31:08.680210 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:08.680215 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:08.680239 (+ 24us) server_negotiation.cc:407] Connection header received
0810 11:31:08.680441 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:08.680445 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:08.680512 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:08.680607 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:31:08.682559 (+ 1952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:08.683293 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:08.684751 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:08.685177 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:08.685286 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:08.688651 (+ 3365us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:31:08.688671 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:31:08.688689 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:31:08.688752 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:31:08.691790 (+ 3038us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:31:08.692644 (+ 854us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:31:08.692659 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:31:08.692662 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:31:08.692823 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:31:08.693272 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:31:08.693277 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:31:08.693282 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:31:08.693791 (+ 509us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:31:08.694020 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0810 11:31:08.694351 (+ 331us) server_negotiation.cc:299] Negotiation successful
0810 11:31:08.694542 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260810 11:31:08.695487 15888 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:31:08.793112 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 15762
I20260810 11:31:08.812038 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 15070
I20260810 11:31:08.813262 15240 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:31:08.934728 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 15070
I20260810 11:31:08.958280 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 15178
I20260810 11:31:08.959666 15436 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:31:09.091367 15971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:09.090628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:36523 (local address 127.0.0.1:51504)
0810 11:31:09.091046 (+ 418us) negotiation.cc:107] Waiting for socket to connect
0810 11:31:09.091168 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.254:36523: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":266}
I20260810 11:31:09.091441 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 15178
2026-08-10T11:31:09Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (10962 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786361401922362-8267-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 11:31:09 dist-test-slave-srgv krb5kdc[16029](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 11:31:09 dist-test-slave-srgv krb5kdc[16029](info): set up 2 sockets
Aug 10 11:31:09 dist-test-slave-srgv krb5kdc[16029](info): commencing operation
krb5kdc: starting...
W20260810 11:31:10.236071 8267 mini_kdc.cc:121] Time spent starting KDC: real 1.094s user 0.004s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786361401922362-8267-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786361401922362-8267-0/krb5kdc/test-user.keytab.
Aug 10 11:31:10 dist-test-slave-srgv krb5kdc[16029](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361470, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T11:31:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T11:31:10Z Disabled control of system clock
WARNING: no policy specified for oryx/127.8.18.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.18.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.18.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786361401922362-8267-0/krb5kdc/oryx_127.8.18.254.keytab.
Entry for principal oryx/127.8.18.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786361401922362-8267-0/krb5kdc/oryx_127.8.18.254.keytab.
I20260810 11:31:10.518762 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786361401922362-8267-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786361401922362-8267-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786361401922362-8267-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.18.254:38029
--webserver_interface=127.8.18.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786361401922362-8267-0/krb5kdc/oryx_127.8.18.254.keytab
--principal=oryx/127.8.18.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.18.212:45989
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.18.254:38029
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786361401922362-8267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786361401922362-8267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786361401922362-8267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:31:10.888537 16042 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:31:10.889063 16042 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:31:10.889274 16042 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:31:10.897753 16042 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 11:31:10.897886 16042 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260810 11:31:10.897930 16042 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:31:10.897987 16042 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:31:10.898048 16042 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 11:31:10.898115 16042 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 11:31:10.909060 16042 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.18.212:45989
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786361401922362-8267-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786361401922362-8267-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.18.254:38029
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786361401922362-8267-0/krb5kdc/oryx_127.8.18.254.keytab
--principal=oryx/127.8.18.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.18.254:38029
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.18.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786361401922362-8267-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:31:10.910548 16042 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:31:10.911943 16042 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:31:10.924980 16047 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:31:10.925045 16050 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:31:10.925074 16048 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:31:10.925977 16042 server_base.cc:1034] running on GCE node
I20260810 11:31:10.926895 16042 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:31:10.928408 16042 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:31:10.929620 16042 hybrid_clock.cc:648] HybridClock initialized: now 1786361470929553 us; error 30 us; skew 500 ppm
I20260810 11:31:10.930008 16042 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:31:10 dist-test-slave-srgv krb5kdc[16029](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361470, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.18.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:31:10.933602 16042 init.cc:377] Logged in from keytab as oryx/127.8.18.254@KRBTEST.COM (short username oryx)
I20260810 11:31:10.935400 16042 webserver.cc:460] Webserver started at http://127.8.18.254:46883/ using document root <none> and password file <none>
I20260810 11:31:10.936264 16042 fs_manager.cc:359] Metadata directory not provided
I20260810 11:31:10.936395 16042 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:31:10.936779 16042 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:31:10.940279 16042 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786361401922362-8267-0/minicluster-data/master-0/data/instance:
uuid: "064ed84ccdb040029c1070bb3cbfe22c"
format_stamp: "Formatted at 2026-08-10 11:31:10 on dist-test-slave-srgv"
server_key: "7a6b0d96cd16e01ee158db04888933cb"
server_key_iv: "c11af87a2280ce6c2cab3ff2db4700a8"
server_key_version: "encryptionkey@0"
I20260810 11:31:10.941303 16042 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786361401922362-8267-0/minicluster-data/master-0/wal/instance:
uuid: "064ed84ccdb040029c1070bb3cbfe22c"
format_stamp: "Formatted at 2026-08-10 11:31:10 on dist-test-slave-srgv"
server_key: "7a6b0d96cd16e01ee158db04888933cb"
server_key_iv: "c11af87a2280ce6c2cab3ff2db4700a8"
server_key_version: "encryptionkey@0"
I20260810 11:31:10.947561 16042 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260810 11:31:10.951668 16057 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:31:10.953647 16042 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260810 11:31:10.953881 16042 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786361401922362-8267-0/minicluster-data/master-0/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786361401922362-8267-0/minicluster-data/master-0/wal
uuid: "064ed84ccdb040029c1070bb3cbfe22c"
format_stamp: "Formatted at 2026-08-10 11:31:10 on dist-test-slave-srgv"
server_key: "7a6b0d96cd16e01ee158db04888933cb"
server_key_iv: "c11af87a2280ce6c2cab3ff2db4700a8"
server_key_version: "encryptionkey@0"
I20260810 11:31:10.954175 16042 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786361401922362-8267-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786361401922362-8267-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786361401922362-8267-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:31:10.980096 16042 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:31:10.985075 16042 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:31:10.985466 16042 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:31:11.010895 16042 rpc_server.cc:307] RPC server started. Bound to: 127.8.18.254:38029
I20260810 11:31:11.010938 16121 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.18.254:38029 every 8 connection(s)
I20260810 11:31:11.012430 16042 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
I20260810 11:31:11.016534 16122 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:31:11.017905 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 16042
I20260810 11:31:11.018177 8267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786361401922362-8267-0/minicluster-data/master-0/wal/instance
I20260810 11:31:11.018870 8267 external_mini_cluster.cc:1428] Setting key 504127bce73cca34cb72f12ea2a319e1
I20260810 11:31:11.033066 16122 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 064ed84ccdb040029c1070bb3cbfe22c: Bootstrap starting.
Aug 10 11:31:11 dist-test-slave-srgv krb5kdc[16029](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361470, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.8.18.254@KRBTEST.COM
I20260810 11:31:11.036948 16122 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 064ed84ccdb040029c1070bb3cbfe22c: Neither blocks nor log segments found. Creating new log.
I20260810 11:31:11.038524 16122 log.cc:824] T 00000000000000000000000000000000 P 064ed84ccdb040029c1070bb3cbfe22c: Log is configured to *not* fsync() on all Append() calls
I20260810 11:31:11.042322 16122 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 064ed84ccdb040029c1070bb3cbfe22c: No bootstrap required, opened a new log
I20260810 11:31:11.046846 16125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:11.021239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50916 (local address 127.8.18.254:38029)
0810 11:31:11.022197 (+ 958us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:11.022214 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:11.022288 (+ 74us) server_negotiation.cc:407] Connection header received
0810 11:31:11.024192 (+ 1904us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:11.024236 (+ 44us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:11.024877 (+ 641us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:11.025445 (+ 568us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:31:11.027095 (+ 1650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:11.029556 (+ 2461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:11.031066 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:11.032249 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:11.032463 (+ 214us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:11.037573 (+ 5110us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:31:11.037588 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:31:11.037607 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:31:11.037680 (+ 73us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:31:11.041359 (+ 3679us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:31:11.042671 (+ 1312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:31:11.042679 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:31:11.042687 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:31:11.042949 (+ 262us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:31:11.043472 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:31:11.043478 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:31:11.043482 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:31:11.044167 (+ 685us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:31:11.044580 (+ 413us) server_negotiation.cc:1035] Waiting for connection context
0810 11:31:11.045116 (+ 536us) server_negotiation.cc:299] Negotiation successful
0810 11:31:11.045691 (+ 575us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":675,"thread_start_us":429,"threads_started":1}
I20260810 11:31:11.051522 16122 raft_consensus.cc:348] T 00000000000000000000000000000000 P 064ed84ccdb040029c1070bb3cbfe22c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "064ed84ccdb040029c1070bb3cbfe22c" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 38029 } }
I20260810 11:31:11.052117 16122 raft_consensus.cc:374] T 00000000000000000000000000000000 P 064ed84ccdb040029c1070bb3cbfe22c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:31:11.052264 16122 raft_consensus.cc:729] T 00000000000000000000000000000000 P 064ed84ccdb040029c1070bb3cbfe22c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 064ed84ccdb040029c1070bb3cbfe22c, State: Initialized, Role: FOLLOWER
I20260810 11:31:11.052958 16122 consensus_queue.cc:260] T 00000000000000000000000000000000 P 064ed84ccdb040029c1070bb3cbfe22c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "064ed84ccdb040029c1070bb3cbfe22c" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 38029 } }
I20260810 11:31:11.053237 16122 raft_consensus.cc:388] T 00000000000000000000000000000000 P 064ed84ccdb040029c1070bb3cbfe22c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 11:31:11.053349 16122 raft_consensus.cc:482] T 00000000000000000000000000000000 P 064ed84ccdb040029c1070bb3cbfe22c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 11:31:11.053578 16122 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 064ed84ccdb040029c1070bb3cbfe22c [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:31:11.055948 16122 raft_consensus.cc:504] T 00000000000000000000000000000000 P 064ed84ccdb040029c1070bb3cbfe22c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "064ed84ccdb040029c1070bb3cbfe22c" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 38029 } }
I20260810 11:31:11.056666 16122 leader_election.cc:302] T 00000000000000000000000000000000 P 064ed84ccdb040029c1070bb3cbfe22c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 064ed84ccdb040029c1070bb3cbfe22c; no voters:
I20260810 11:31:11.057364 16122 leader_election.cc:288] T 00000000000000000000000000000000 P 064ed84ccdb040029c1070bb3cbfe22c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 11:31:11.057586 16127 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 064ed84ccdb040029c1070bb3cbfe22c [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:31:11.059330 16127 raft_consensus.cc:686] T 00000000000000000000000000000000 P 064ed84ccdb040029c1070bb3cbfe22c [term 1 LEADER]: Becoming Leader. State: Replica: 064ed84ccdb040029c1070bb3cbfe22c, State: Running, Role: LEADER
I20260810 11:31:11.060015 16127 consensus_queue.cc:237] T 00000000000000000000000000000000 P 064ed84ccdb040029c1070bb3cbfe22c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "064ed84ccdb040029c1070bb3cbfe22c" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 38029 } }
I20260810 11:31:11.060190 16122 sys_catalog.cc:564] T 00000000000000000000000000000000 P 064ed84ccdb040029c1070bb3cbfe22c [sys.catalog]: configured and running, proceeding with master startup.
I20260810 11:31:11.063194 16128 sys_catalog.cc:455] T 00000000000000000000000000000000 P 064ed84ccdb040029c1070bb3cbfe22c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "064ed84ccdb040029c1070bb3cbfe22c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "064ed84ccdb040029c1070bb3cbfe22c" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 38029 } } }
I20260810 11:31:11.063692 16128 sys_catalog.cc:458] T 00000000000000000000000000000000 P 064ed84ccdb040029c1070bb3cbfe22c [sys.catalog]: This master's current role is: LEADER
I20260810 11:31:11.063639 16129 sys_catalog.cc:455] T 00000000000000000000000000000000 P 064ed84ccdb040029c1070bb3cbfe22c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 064ed84ccdb040029c1070bb3cbfe22c. Latest consensus state: current_term: 1 leader_uuid: "064ed84ccdb040029c1070bb3cbfe22c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "064ed84ccdb040029c1070bb3cbfe22c" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 38029 } } }
I20260810 11:31:11.063997 16129 sys_catalog.cc:458] T 00000000000000000000000000000000 P 064ed84ccdb040029c1070bb3cbfe22c [sys.catalog]: This master's current role is: LEADER
I20260810 11:31:11.065117 16136 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 11:31:11.072993 16136 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 11:31:11.084158 16136 catalog_manager.cc:1347] Generated new cluster ID: 93e559b958664515846503780cdab048
I20260810 11:31:11.084329 16136 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 11:31:11.104700 16136 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 11:31:11.106511 16136 catalog_manager.cc:1504] Loading token signing keys...
I20260810 11:31:11.122006 16136 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 064ed84ccdb040029c1070bb3cbfe22c: Generated new TSK 0
I20260810 11:31:11.123312 16136 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.8.18.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.18.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.18.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786361401922362-8267-0/krb5kdc/oryx_127.8.18.193.keytab.
Entry for principal oryx/127.8.18.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786361401922362-8267-0/krb5kdc/oryx_127.8.18.193.keytab.
I20260810 11:31:11.189826 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786361401922362-8267-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786361401922362-8267-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786361401922362-8267-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786361401922362-8267-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.18.193:0
--local_ip_for_outbound_sockets=127.8.18.193
--webserver_interface=127.8.18.193
--webserver_port=0
--tserver_master_addrs=127.8.18.254:38029
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786361401922362-8267-0/krb5kdc/oryx_127.8.18.193.keytab
--principal=oryx/127.8.18.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.18.212:45989
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786361401922362-8267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786361401922362-8267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786361401922362-8267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:31:11.561585 16148 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:31:11.561949 16148 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:31:11.562171 16148 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:31:11.570348 16148 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:31:11.570616 16148 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:31:11.570827 16148 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 11:31:11.570982 16148 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.18.193
I20260810 11:31:11.582180 16148 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.18.212:45989
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786361401922362-8267-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786361401922362-8267-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786361401922362-8267-0/krb5kdc/oryx_127.8.18.193.keytab
--principal=oryx/127.8.18.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.18.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786361401922362-8267-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.18.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.18.254:38029
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.18.193
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786361401922362-8267-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:31:11.583645 16148 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:31:11.585456 16148 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:31:11.599784 16153 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:31:11.599812 16156 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:31:11.599737 16154 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:31:11.600905 16148 server_base.cc:1034] running on GCE node
I20260810 11:31:11.601692 16148 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:31:11.602694 16148 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:31:11.603955 16148 hybrid_clock.cc:648] HybridClock initialized: now 1786361471603872 us; error 65 us; skew 500 ppm
I20260810 11:31:11.604337 16148 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:31:11 dist-test-slave-srgv krb5kdc[16029](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361471, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.18.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:31:11.608335 16148 init.cc:377] Logged in from keytab as oryx/127.8.18.193@KRBTEST.COM (short username oryx)
I20260810 11:31:11.610213 16148 webserver.cc:460] Webserver started at http://127.8.18.193:37167/ using document root <none> and password file <none>
I20260810 11:31:11.611042 16148 fs_manager.cc:359] Metadata directory not provided
I20260810 11:31:11.611172 16148 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:31:11.611578 16148 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:31:11.614969 16148 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786361401922362-8267-0/minicluster-data/ts-0/data/instance:
uuid: "85786fb38f3f4634ae1b440c35d07d3a"
format_stamp: "Formatted at 2026-08-10 11:31:11 on dist-test-slave-srgv"
server_key: "15629dec19517e646550e372354b15e1"
server_key_iv: "ed36e3d6ab1b02e29d93c41dc6eb4b0b"
server_key_version: "encryptionkey@0"
I20260810 11:31:11.615918 16148 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786361401922362-8267-0/minicluster-data/ts-0/wal/instance:
uuid: "85786fb38f3f4634ae1b440c35d07d3a"
format_stamp: "Formatted at 2026-08-10 11:31:11 on dist-test-slave-srgv"
server_key: "15629dec19517e646550e372354b15e1"
server_key_iv: "ed36e3d6ab1b02e29d93c41dc6eb4b0b"
server_key_version: "encryptionkey@0"
I20260810 11:31:11.622169 16148 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260810 11:31:11.626438 16163 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:31:11.628057 16148 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.005s sys 0.000s
I20260810 11:31:11.628297 16148 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786361401922362-8267-0/minicluster-data/ts-0/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786361401922362-8267-0/minicluster-data/ts-0/wal
uuid: "85786fb38f3f4634ae1b440c35d07d3a"
format_stamp: "Formatted at 2026-08-10 11:31:11 on dist-test-slave-srgv"
server_key: "15629dec19517e646550e372354b15e1"
server_key_iv: "ed36e3d6ab1b02e29d93c41dc6eb4b0b"
server_key_version: "encryptionkey@0"
I20260810 11:31:11.628530 16148 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786361401922362-8267-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786361401922362-8267-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786361401922362-8267-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:31:11.662310 16148 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:31:11.667441 16148 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:31:11.667765 16148 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:31:11.670893 16148 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:31:11.671000 16148 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:31:11.671218 16148 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:31:11.671330 16148 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 11:31:11 dist-test-slave-srgv krb5kdc[16029](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361471, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.18.193@KRBTEST.COM for oryx/127.8.18.254@KRBTEST.COM
I20260810 11:31:11.699424 16173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:11.671567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.193:42825 (local address 127.8.18.254:38029)
0810 11:31:11.672345 (+ 778us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:11.672352 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:11.673464 (+ 1112us) server_negotiation.cc:407] Connection header received
0810 11:31:11.675652 (+ 2188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:11.675658 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:11.675771 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:11.675899 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:31:11.678619 (+ 2720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:11.679778 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:11.683679 (+ 3901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:11.684231 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:11.684401 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:11.691007 (+ 6606us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:31:11.691017 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:31:11.691024 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:31:11.691077 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:31:11.694311 (+ 3234us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:31:11.695809 (+ 1498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:31:11.695815 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:31:11.695818 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:31:11.695968 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:31:11.696737 (+ 769us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:31:11.696742 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:31:11.696744 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:31:11.697097 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:31:11.697304 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0810 11:31:11.698662 (+ 1358us) server_negotiation.cc:299] Negotiation successful
0810 11:31:11.698995 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":637,"thread_start_us":510,"threads_started":1}
I20260810 11:31:11.701548 16174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:11.672310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:38029 (local address 127.8.18.193:42825)
0810 11:31:11.673238 (+ 928us) negotiation.cc:107] Waiting for socket to connect
0810 11:31:11.673299 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0810 11:31:11.675323 (+ 2024us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:31:11.676246 (+ 923us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:31:11.676261 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:31:11.677285 (+ 1024us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:31:11.678361 (+ 1076us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:11.678406 (+ 45us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:11.680060 (+ 1654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:11.680069 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:11.683388 (+ 3319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:11.683425 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:11.684607 (+ 1182us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:11.684619 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:11.685077 (+ 458us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:11.686157 (+ 1080us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:31:11.686207 (+ 50us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:31:11.690565 (+ 4358us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:31:11.694556 (+ 3991us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:31:11.694574 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:31:11.694600 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:31:11.695581 (+ 981us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:31:11.696116 (+ 535us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:31:11.696125 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:31:11.696131 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:31:11.696540 (+ 409us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:31:11.697257 (+ 717us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:31:11.697270 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:31:11.697997 (+ 727us) client_negotiation.cc:769] Sending connection context
0810 11:31:11.698432 (+ 435us) client_negotiation.cc:240] Negotiation successful
0810 11:31:11.699358 (+ 926us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":625,"thread_start_us":394,"threads_started":1}
I20260810 11:31:11.727064 16248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:11.708845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:38029 (local address 127.8.18.193:43977)
0810 11:31:11.709768 (+ 923us) negotiation.cc:107] Waiting for socket to connect
0810 11:31:11.709789 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 11:31:11.710011 (+ 222us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:31:11.710506 (+ 495us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:31:11.710523 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:31:11.711509 (+ 986us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:31:11.712186 (+ 677us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:11.712209 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:11.714084 (+ 1875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:11.714093 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:11.715561 (+ 1468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:11.715580 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:11.716704 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:11.716718 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:11.717007 (+ 289us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:11.717958 (+ 951us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:31:11.717993 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:31:11.719525 (+ 1532us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:31:11.723389 (+ 3864us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:31:11.723398 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:31:11.723405 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:31:11.724249 (+ 844us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:31:11.724791 (+ 542us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:31:11.724796 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:31:11.724800 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:31:11.724981 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:31:11.725867 (+ 886us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:31:11.725874 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:31:11.726174 (+ 300us) client_negotiation.cc:769] Sending connection context
0810 11:31:11.726328 (+ 154us) client_negotiation.cc:240] Negotiation successful
0810 11:31:11.726623 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":714,"thread_start_us":479,"threads_started":1}
I20260810 11:31:11.727197 16173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:11.708921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.193:43977 (local address 127.8.18.254:38029)
0810 11:31:11.709192 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:11.709203 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:11.709913 (+ 710us) server_negotiation.cc:407] Connection header received
0810 11:31:11.710178 (+ 265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:11.710183 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:11.710275 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:11.710412 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:31:11.712488 (+ 2076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:11.713879 (+ 1391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:11.715749 (+ 1870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:11.716432 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:11.716628 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:11.719834 (+ 3206us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:31:11.719839 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:31:11.719843 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:31:11.719892 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:31:11.723200 (+ 3308us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:31:11.724424 (+ 1224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:31:11.724433 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:31:11.724437 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:31:11.724646 (+ 209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:31:11.725168 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:31:11.725176 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:31:11.725181 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:31:11.725700 (+ 519us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:31:11.725885 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0810 11:31:11.726475 (+ 590us) server_negotiation.cc:299] Negotiation successful
0810 11:31:11.726761 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260810 11:31:11.730662 16148 rpc_server.cc:307] RPC server started. Bound to: 127.8.18.193:41221
I20260810 11:31:11.730757 16288 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.18.193:41221 every 8 connection(s)
I20260810 11:31:11.732034 16148 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786361401922362-8267-0/minicluster-data/ts-0/data/info.pb
I20260810 11:31:11.734328 16290 heartbeater.cc:347] Connected to a master server at 127.8.18.254:38029
I20260810 11:31:11.734582 16290 heartbeater.cc:464] Registering TS with master...
I20260810 11:31:11.735273 16290 heartbeater.cc:511] Master 127.8.18.254:38029 requested a full tablet report, sending...
I20260810 11:31:11.737525 16075 ts_manager.cc:194] Registered new tserver with Master: 85786fb38f3f4634ae1b440c35d07d3a (127.8.18.193:41221)
I20260810 11:31:11.738519 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 16148
I20260810 11:31:11.738746 8267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786361401922362-8267-0/minicluster-data/ts-0/wal/instance
I20260810 11:31:11.739197 16075 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.18.193@KRBTEST.COM'} at 127.8.18.193:42825
I20260810 11:31:11.739444 8267 external_mini_cluster.cc:1428] Setting key 3f48b7c6337b544e4f7ac9581f613fcb
WARNING: no policy specified for oryx/127.8.18.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.18.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.18.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786361401922362-8267-0/krb5kdc/oryx_127.8.18.194.keytab.
Entry for principal oryx/127.8.18.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786361401922362-8267-0/krb5kdc/oryx_127.8.18.194.keytab.
I20260810 11:31:11.797859 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786361401922362-8267-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786361401922362-8267-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786361401922362-8267-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786361401922362-8267-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.18.194:0
--local_ip_for_outbound_sockets=127.8.18.194
--webserver_interface=127.8.18.194
--webserver_port=0
--tserver_master_addrs=127.8.18.254:38029
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786361401922362-8267-0/krb5kdc/oryx_127.8.18.194.keytab
--principal=oryx/127.8.18.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.18.212:45989
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786361401922362-8267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786361401922362-8267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786361401922362-8267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:31:12.171849 16294 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:31:12.172204 16294 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:31:12.172359 16294 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:31:12.180215 16294 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:31:12.180368 16294 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:31:12.180521 16294 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 11:31:12.180629 16294 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.18.194
I20260810 11:31:12.191478 16294 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.18.212:45989
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786361401922362-8267-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786361401922362-8267-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786361401922362-8267-0/krb5kdc/oryx_127.8.18.194.keytab
--principal=oryx/127.8.18.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.18.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786361401922362-8267-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.18.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.18.254:38029
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.18.194
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786361401922362-8267-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:31:12.192943 16294 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:31:12.194470 16294 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:31:12.209230 16299 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:31:12.209133 16300 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:31:12.209183 16302 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:31:12.210314 16294 server_base.cc:1034] running on GCE node
I20260810 11:31:12.211037 16294 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:31:12.212102 16294 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:31:12.213375 16294 hybrid_clock.cc:648] HybridClock initialized: now 1786361472213298 us; error 48 us; skew 500 ppm
I20260810 11:31:12.213759 16294 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:31:12 dist-test-slave-srgv krb5kdc[16029](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361472, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.18.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:31:12.220793 16294 init.cc:377] Logged in from keytab as oryx/127.8.18.194@KRBTEST.COM (short username oryx)
I20260810 11:31:12.222591 16294 webserver.cc:460] Webserver started at http://127.8.18.194:46667/ using document root <none> and password file <none>
I20260810 11:31:12.223400 16294 fs_manager.cc:359] Metadata directory not provided
I20260810 11:31:12.223544 16294 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:31:12.223896 16294 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:31:12.227212 16294 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786361401922362-8267-0/minicluster-data/ts-1/data/instance:
uuid: "83fb5fdfb5e34ed399aad0e87934d93c"
format_stamp: "Formatted at 2026-08-10 11:31:12 on dist-test-slave-srgv"
server_key: "cc92f2480997530194d797ae5db4c478"
server_key_iv: "4ec5d9e33c432d53c56631afd294b00f"
server_key_version: "encryptionkey@0"
I20260810 11:31:12.228260 16294 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786361401922362-8267-0/minicluster-data/ts-1/wal/instance:
uuid: "83fb5fdfb5e34ed399aad0e87934d93c"
format_stamp: "Formatted at 2026-08-10 11:31:12 on dist-test-slave-srgv"
server_key: "cc92f2480997530194d797ae5db4c478"
server_key_iv: "4ec5d9e33c432d53c56631afd294b00f"
server_key_version: "encryptionkey@0"
I20260810 11:31:12.234454 16294 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260810 11:31:12.238572 16309 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:31:12.240183 16294 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260810 11:31:12.240406 16294 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786361401922362-8267-0/minicluster-data/ts-1/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786361401922362-8267-0/minicluster-data/ts-1/wal
uuid: "83fb5fdfb5e34ed399aad0e87934d93c"
format_stamp: "Formatted at 2026-08-10 11:31:12 on dist-test-slave-srgv"
server_key: "cc92f2480997530194d797ae5db4c478"
server_key_iv: "4ec5d9e33c432d53c56631afd294b00f"
server_key_version: "encryptionkey@0"
I20260810 11:31:12.240679 16294 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786361401922362-8267-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786361401922362-8267-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786361401922362-8267-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:31:12.278367 16294 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:31:12.283674 16294 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:31:12.284061 16294 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:31:12.287408 16294 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:31:12.287551 16294 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:31:12.287740 16294 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:31:12.287822 16294 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 11:31:12 dist-test-slave-srgv krb5kdc[16029](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361472, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.18.194@KRBTEST.COM for oryx/127.8.18.254@KRBTEST.COM
I20260810 11:31:12.318343 16319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:12.288511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.194:56051 (local address 127.8.18.254:38029)
0810 11:31:12.289078 (+ 567us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:12.289086 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:12.290859 (+ 1773us) server_negotiation.cc:407] Connection header received
0810 11:31:12.294644 (+ 3785us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:12.294657 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:12.294817 (+ 160us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:12.294969 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:31:12.297932 (+ 2963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:12.299175 (+ 1243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:12.301774 (+ 2599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:12.302660 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:12.302865 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:12.309264 (+ 6399us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:31:12.309271 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:31:12.309275 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:31:12.309322 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:31:12.313219 (+ 3897us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:31:12.314777 (+ 1558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:31:12.314786 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:31:12.314790 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:31:12.314995 (+ 205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:31:12.315731 (+ 736us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:31:12.315738 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:31:12.315742 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:31:12.316070 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:31:12.316288 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0810 11:31:12.317538 (+ 1250us) server_negotiation.cc:299] Negotiation successful
0810 11:31:12.317865 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":409,"thread_start_us":262,"threads_started":1}
I20260810 11:31:12.320003 16321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:12.289360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:38029 (local address 127.8.18.194:56051)
0810 11:31:12.290595 (+ 1235us) negotiation.cc:107] Waiting for socket to connect
0810 11:31:12.290660 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0810 11:31:12.294061 (+ 3401us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:31:12.295399 (+ 1338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:31:12.295413 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:31:12.296473 (+ 1060us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:31:12.297552 (+ 1079us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:12.297594 (+ 42us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:12.299396 (+ 1802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:12.299400 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:12.301496 (+ 2096us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:12.301514 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:12.303025 (+ 1511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:12.303033 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:12.303298 (+ 265us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:12.304408 (+ 1110us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:31:12.304447 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:31:12.308838 (+ 4391us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:31:12.313506 (+ 4668us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:31:12.313532 (+ 26us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:31:12.313565 (+ 33us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:31:12.314543 (+ 978us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:31:12.315156 (+ 613us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:31:12.315165 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:31:12.315170 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:31:12.315497 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:31:12.316258 (+ 761us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:31:12.316272 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:31:12.316894 (+ 622us) client_negotiation.cc:769] Sending connection context
0810 11:31:12.317359 (+ 465us) client_negotiation.cc:240] Negotiation successful
0810 11:31:12.318120 (+ 761us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":877,"thread_start_us":572,"threads_started":1}
I20260810 11:31:12.346910 16399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:12.327286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:38029 (local address 127.8.18.194:40893)
0810 11:31:12.328120 (+ 834us) negotiation.cc:107] Waiting for socket to connect
0810 11:31:12.328171 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0810 11:31:12.328395 (+ 224us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:31:12.328820 (+ 425us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:31:12.328826 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:31:12.329463 (+ 637us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:31:12.329901 (+ 438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:12.329923 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:12.331729 (+ 1806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:12.331736 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:12.333842 (+ 2106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:12.333862 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:12.335091 (+ 1229us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:12.335097 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:12.335242 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:12.336278 (+ 1036us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:31:12.336311 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:31:12.337660 (+ 1349us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:31:12.342417 (+ 4757us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:31:12.342429 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:31:12.342434 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:31:12.343035 (+ 601us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:31:12.343663 (+ 628us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:31:12.343672 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:31:12.343680 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:31:12.343950 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:31:12.345172 (+ 1222us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:31:12.345180 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:31:12.345902 (+ 722us) client_negotiation.cc:769] Sending connection context
0810 11:31:12.346104 (+ 202us) client_negotiation.cc:240] Negotiation successful
0810 11:31:12.346498 (+ 394us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":660,"thread_start_us":480,"threads_started":1}
I20260810 11:31:12.347152 16319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:12.327238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.194:40893 (local address 127.8.18.254:38029)
0810 11:31:12.327427 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:12.327432 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:12.328308 (+ 876us) server_negotiation.cc:407] Connection header received
0810 11:31:12.328570 (+ 262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:12.328575 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:12.328663 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:12.328783 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:31:12.330171 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:12.331490 (+ 1319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:12.334071 (+ 2581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:12.334838 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:12.335025 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:12.337915 (+ 2890us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:31:12.337921 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:31:12.337924 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:31:12.337964 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:31:12.342168 (+ 4204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:31:12.343259 (+ 1091us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:31:12.343264 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:31:12.343268 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:31:12.343455 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:31:12.344190 (+ 735us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:31:12.344199 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:31:12.344203 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:31:12.344930 (+ 727us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:31:12.345212 (+ 282us) server_negotiation.cc:1035] Waiting for connection context
0810 11:31:12.346295 (+ 1083us) server_negotiation.cc:299] Negotiation successful
0810 11:31:12.346633 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260810 11:31:12.349311 16294 rpc_server.cc:307] RPC server started. Bound to: 127.8.18.194:45817
I20260810 11:31:12.349346 16435 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.18.194:45817 every 8 connection(s)
I20260810 11:31:12.350976 16294 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786361401922362-8267-0/minicluster-data/ts-1/data/info.pb
I20260810 11:31:12.353577 16436 heartbeater.cc:347] Connected to a master server at 127.8.18.254:38029
I20260810 11:31:12.353832 16436 heartbeater.cc:464] Registering TS with master...
I20260810 11:31:12.354542 16436 heartbeater.cc:511] Master 127.8.18.254:38029 requested a full tablet report, sending...
I20260810 11:31:12.355937 16075 ts_manager.cc:194] Registered new tserver with Master: 83fb5fdfb5e34ed399aad0e87934d93c (127.8.18.194:45817)
I20260810 11:31:12.356905 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 16294
I20260810 11:31:12.357131 8267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786361401922362-8267-0/minicluster-data/ts-1/wal/instance
I20260810 11:31:12.357081 16075 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.18.194@KRBTEST.COM'} at 127.8.18.194:56051
I20260810 11:31:12.357766 8267 external_mini_cluster.cc:1428] Setting key e6b8d86223bd792bbefdbd84779eee52
WARNING: no policy specified for oryx/127.8.18.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.18.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.18.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786361401922362-8267-0/krb5kdc/oryx_127.8.18.195.keytab.
Entry for principal oryx/127.8.18.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786361401922362-8267-0/krb5kdc/oryx_127.8.18.195.keytab.
I20260810 11:31:12.417397 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786361401922362-8267-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786361401922362-8267-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786361401922362-8267-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786361401922362-8267-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.18.195:0
--local_ip_for_outbound_sockets=127.8.18.195
--webserver_interface=127.8.18.195
--webserver_port=0
--tserver_master_addrs=127.8.18.254:38029
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786361401922362-8267-0/krb5kdc/oryx_127.8.18.195.keytab
--principal=oryx/127.8.18.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.18.212:45989
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786361401922362-8267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786361401922362-8267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786361401922362-8267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 11:31:12.742818 16290 heartbeater.cc:503] Master 127.8.18.254:38029 was elected leader, sending a full tablet report...
W20260810 11:31:12.777087 16440 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:31:12.777474 16440 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:31:12.777652 16440 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:31:12.785372 16440 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:31:12.785617 16440 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:31:12.785763 16440 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 11:31:12.785900 16440 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.18.195
I20260810 11:31:12.796995 16440 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.18.212:45989
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786361401922362-8267-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786361401922362-8267-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786361401922362-8267-0/krb5kdc/oryx_127.8.18.195.keytab
--principal=oryx/127.8.18.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.18.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786361401922362-8267-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.18.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.18.254:38029
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.18.195
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786361401922362-8267-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:31:12.798444 16440 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:31:12.799860 16440 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:31:12.814719 16445 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:31:12.814939 16448 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:31:12.814858 16446 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:31:12.816335 16440 server_base.cc:1034] running on GCE node
I20260810 11:31:12.817207 16440 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:31:12.818416 16440 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:31:12.819751 16440 hybrid_clock.cc:648] HybridClock initialized: now 1786361472819669 us; error 50 us; skew 500 ppm
I20260810 11:31:12.820187 16440 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:31:12 dist-test-slave-srgv krb5kdc[16029](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361472, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.18.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:31:12.824326 16440 init.cc:377] Logged in from keytab as oryx/127.8.18.195@KRBTEST.COM (short username oryx)
I20260810 11:31:12.826234 16440 webserver.cc:460] Webserver started at http://127.8.18.195:33443/ using document root <none> and password file <none>
I20260810 11:31:12.827083 16440 fs_manager.cc:359] Metadata directory not provided
I20260810 11:31:12.827240 16440 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:31:12.827613 16440 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:31:12.831048 16440 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786361401922362-8267-0/minicluster-data/ts-2/data/instance:
uuid: "fc6dfd46f9cc4201886c620db1510312"
format_stamp: "Formatted at 2026-08-10 11:31:12 on dist-test-slave-srgv"
server_key: "fd756644bfeb8b459545a1fc4bc57834"
server_key_iv: "b4c2b1b9381134c8a3a0853a4de693bd"
server_key_version: "encryptionkey@0"
I20260810 11:31:12.831964 16440 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786361401922362-8267-0/minicluster-data/ts-2/wal/instance:
uuid: "fc6dfd46f9cc4201886c620db1510312"
format_stamp: "Formatted at 2026-08-10 11:31:12 on dist-test-slave-srgv"
server_key: "fd756644bfeb8b459545a1fc4bc57834"
server_key_iv: "b4c2b1b9381134c8a3a0853a4de693bd"
server_key_version: "encryptionkey@0"
I20260810 11:31:12.838589 16440 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.004s
I20260810 11:31:12.842820 16456 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:31:12.844604 16440 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260810 11:31:12.844816 16440 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786361401922362-8267-0/minicluster-data/ts-2/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786361401922362-8267-0/minicluster-data/ts-2/wal
uuid: "fc6dfd46f9cc4201886c620db1510312"
format_stamp: "Formatted at 2026-08-10 11:31:12 on dist-test-slave-srgv"
server_key: "fd756644bfeb8b459545a1fc4bc57834"
server_key_iv: "b4c2b1b9381134c8a3a0853a4de693bd"
server_key_version: "encryptionkey@0"
I20260810 11:31:12.845085 16440 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786361401922362-8267-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786361401922362-8267-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786361401922362-8267-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:31:12.861613 16440 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:31:12.866454 16440 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:31:12.866719 16440 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:31:12.869681 16440 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:31:12.869822 16440 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:31:12.869988 16440 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:31:12.870066 16440 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 11:31:12 dist-test-slave-srgv krb5kdc[16029](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361472, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.18.195@KRBTEST.COM for oryx/127.8.18.254@KRBTEST.COM
I20260810 11:31:12.901149 16466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:12.870642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.195:56093 (local address 127.8.18.254:38029)
0810 11:31:12.873599 (+ 2957us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:12.873607 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:12.873635 (+ 28us) server_negotiation.cc:407] Connection header received
0810 11:31:12.876192 (+ 2557us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:12.876199 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:12.876321 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:12.876465 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:31:12.879899 (+ 3434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:12.880981 (+ 1082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:12.885362 (+ 4381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:12.886140 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:12.886330 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:12.892484 (+ 6154us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:31:12.892494 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:31:12.892501 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:31:12.892563 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:31:12.896221 (+ 3658us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:31:12.897578 (+ 1357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:31:12.897586 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:31:12.897589 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:31:12.897817 (+ 228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:31:12.898386 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:31:12.898394 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:31:12.898400 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:31:12.898966 (+ 566us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:31:12.899195 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0810 11:31:12.900094 (+ 899us) server_negotiation.cc:299] Negotiation successful
0810 11:31:12.900465 (+ 371us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2809,"thread_start_us":2676,"threads_started":1}
I20260810 11:31:12.901718 16467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:12.871412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:38029 (local address 127.8.18.195:56093)
0810 11:31:12.872710 (+ 1298us) negotiation.cc:107] Waiting for socket to connect
0810 11:31:12.872813 (+ 103us) client_negotiation.cc:174] Beginning negotiation
0810 11:31:12.875639 (+ 2826us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:31:12.876874 (+ 1235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:31:12.876893 (+ 19us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:31:12.878110 (+ 1217us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:31:12.879563 (+ 1453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:12.879605 (+ 42us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:12.881249 (+ 1644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:12.881257 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:12.884985 (+ 3728us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:12.885021 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:12.886477 (+ 1456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:12.886487 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:12.886881 (+ 394us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:12.887894 (+ 1013us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:31:12.887924 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:31:12.892081 (+ 4157us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:31:12.896425 (+ 4344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:31:12.896441 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:31:12.896458 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:31:12.897263 (+ 805us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:31:12.897991 (+ 728us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:31:12.897995 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:31:12.897998 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:31:12.898222 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:31:12.899165 (+ 943us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:31:12.899173 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:31:12.899588 (+ 415us) client_negotiation.cc:769] Sending connection context
0810 11:31:12.899865 (+ 277us) client_negotiation.cc:240] Negotiation successful
0810 11:31:12.900326 (+ 461us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":831,"thread_start_us":475,"threads_started":1}
I20260810 11:31:12.918998 16440 rpc_server.cc:307] RPC server started. Bound to: 127.8.18.195:35295
I20260810 11:31:12.919041 16580 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.18.195:35295 every 8 connection(s)
I20260810 11:31:12.920465 16440 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786361401922362-8267-0/minicluster-data/ts-2/data/info.pb
I20260810 11:31:12.923197 16581 heartbeater.cc:347] Connected to a master server at 127.8.18.254:38029
I20260810 11:31:12.923439 16581 heartbeater.cc:464] Registering TS with master...
I20260810 11:31:12.924278 16581 heartbeater.cc:511] Master 127.8.18.254:38029 requested a full tablet report, sending...
I20260810 11:31:12.925595 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 16440
I20260810 11:31:12.925809 8267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786361401922362-8267-0/minicluster-data/ts-2/wal/instance
I20260810 11:31:12.926340 16075 ts_manager.cc:194] Registered new tserver with Master: fc6dfd46f9cc4201886c620db1510312 (127.8.18.195:35295)
I20260810 11:31:12.926366 8267 external_mini_cluster.cc:1428] Setting key d75f4c6e95c1a16fbf6f8bd661ef521e
I20260810 11:31:12.927491 16075 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.18.195@KRBTEST.COM'} at 127.8.18.195:56093
I20260810 11:31:12.927809 16560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:12.910254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:38029 (local address 127.8.18.195:41729)
0810 11:31:12.911328 (+ 1074us) negotiation.cc:107] Waiting for socket to connect
0810 11:31:12.911349 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 11:31:12.911559 (+ 210us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:31:12.912212 (+ 653us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:31:12.912216 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:31:12.913015 (+ 799us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:31:12.913675 (+ 660us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:12.913690 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:12.915271 (+ 1581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:12.915281 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:12.917336 (+ 2055us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:12.917362 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:12.918485 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:12.918492 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:12.918675 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:12.919407 (+ 732us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:31:12.919427 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:31:12.920245 (+ 818us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:31:12.924484 (+ 4239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:31:12.924491 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:31:12.924497 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:31:12.925130 (+ 633us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:31:12.925717 (+ 587us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:31:12.925725 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:31:12.925730 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:31:12.925880 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:31:12.926609 (+ 729us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:31:12.926614 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:31:12.926815 (+ 201us) client_negotiation.cc:769] Sending connection context
0810 11:31:12.926978 (+ 163us) client_negotiation.cc:240] Negotiation successful
0810 11:31:12.927330 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":851,"thread_start_us":640,"threads_started":1}
I20260810 11:31:12.928774 16466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:12.910157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.195:41729 (local address 127.8.18.254:38029)
0810 11:31:12.910423 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:12.910433 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:12.911471 (+ 1038us) server_negotiation.cc:407] Connection header received
0810 11:31:12.911724 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:12.911740 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:12.911853 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:12.911977 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:31:12.913893 (+ 1916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:12.915062 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:12.917600 (+ 2538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:12.918224 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:12.918389 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:12.920499 (+ 2110us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:31:12.920505 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:31:12.920509 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:31:12.920574 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:31:12.924288 (+ 3714us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:31:12.925318 (+ 1030us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:31:12.925327 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:31:12.925331 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:31:12.925565 (+ 234us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:31:12.926008 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:31:12.926013 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:31:12.926016 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:31:12.926448 (+ 432us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:31:12.926645 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0810 11:31:12.928155 (+ 1510us) server_negotiation.cc:299] Negotiation successful
0810 11:31:12.928429 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260810 11:31:12.929958 8267 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 10 11:31:12 dist-test-slave-srgv krb5kdc[16029](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361472, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 10 11:31:12 dist-test-slave-srgv krb5kdc[16029](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.8.18.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 11:31:12 dist-test-slave-srgv krb5kdc[16029](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.8.18.254@KRBTEST.COM, Server not found in Kerberos database
W20260810 11:31:12.992308 16466 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:31:12.980313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50920 (local address 127.8.18.254:38029)
0810 11:31:12.980588 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:12.980594 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:12.980677 (+ 83us) server_negotiation.cc:407] Connection header received
0810 11:31:12.980999 (+ 322us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:12.981012 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:12.981099 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:12.981210 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:31:12.982806 (+ 1596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:12.983566 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:12.984993 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:12.985592 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:12.985708 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:12.991920 (+ 6212us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50920: BlockingRecv error: recv got EOF from 127.0.0.1:50920 (error 108)
Metrics: {"server-negotiator.queue_time_us":107}
Aug 10 11:31:13 dist-test-slave-srgv krb5kdc[16029](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361472, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.8.18.254@KRBTEST.COM
I20260810 11:31:13.011859 16466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:12.996161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50930 (local address 127.8.18.254:38029)
0810 11:31:12.996378 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:12.996384 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:12.996711 (+ 327us) server_negotiation.cc:407] Connection header received
0810 11:31:12.996889 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:12.996897 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:12.997003 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:12.997134 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:31:12.998475 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:12.999210 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:13.000511 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:13.001004 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:13.001124 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:13.005884 (+ 4760us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:31:13.005890 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:31:13.005894 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:31:13.005944 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:31:13.008769 (+ 2825us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:31:13.009648 (+ 879us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:31:13.009653 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:31:13.009655 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:31:13.009794 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:31:13.010225 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:31:13.010230 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:31:13.010240 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:31:13.010686 (+ 446us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:31:13.010922 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0810 11:31:13.011336 (+ 414us) server_negotiation.cc:299] Negotiation successful
0810 11:31:13.011560 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260810 11:31:13.017035 16075 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50930:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 11:31:13.025110 16075 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 11:31:13.051232 16601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:13.039448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.193:41221 (local address 127.0.0.1:36068)
0810 11:31:13.040816 (+ 1368us) negotiation.cc:107] Waiting for socket to connect
0810 11:31:13.040855 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0810 11:31:13.041059 (+ 204us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:31:13.042288 (+ 1229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:31:13.042296 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:31:13.042338 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:31:13.042812 (+ 474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:13.042826 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:13.044957 (+ 2131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:13.044963 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:13.047664 (+ 2701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:13.047683 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:13.049878 (+ 2195us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:13.049887 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:13.050118 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:13.050205 (+ 87us) client_negotiation.cc:769] Sending connection context
0810 11:31:13.050545 (+ 340us) client_negotiation.cc:240] Negotiation successful
0810 11:31:13.050797 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1106,"thread_start_us":943,"threads_started":1}
I20260810 11:31:13.051860 16602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:13.039630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36068 (local address 127.8.18.193:41221)
0810 11:31:13.040328 (+ 698us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:13.040336 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:13.040977 (+ 641us) server_negotiation.cc:407] Connection header received
0810 11:31:13.041343 (+ 366us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:13.041356 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:13.041695 (+ 339us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:13.041912 (+ 217us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:31:13.043036 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:13.044741 (+ 1705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:13.047864 (+ 3123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:13.049522 (+ 1658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:13.049738 (+ 216us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:13.050031 (+ 293us) server_negotiation.cc:1035] Waiting for connection context
0810 11:31:13.050570 (+ 539us) server_negotiation.cc:299] Negotiation successful
0810 11:31:13.050788 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":502,"thread_start_us":335,"threads_started":1}
I20260810 11:31:13.052968 16603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:13.041243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.194:45817 (local address 127.0.0.1:51318)
0810 11:31:13.041917 (+ 674us) negotiation.cc:107] Waiting for socket to connect
0810 11:31:13.041937 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 11:31:13.042050 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:31:13.044686 (+ 2636us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:31:13.044701 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:31:13.044733 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:31:13.045257 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:13.045271 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:13.047221 (+ 1950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:13.047237 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:13.049801 (+ 2564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:13.049819 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:13.052121 (+ 2302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:13.052128 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:13.052324 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:13.052359 (+ 35us) client_negotiation.cc:769] Sending connection context
0810 11:31:13.052430 (+ 71us) client_negotiation.cc:240] Negotiation successful
0810 11:31:13.052576 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":506,"mutex_wait_us":61,"thread_start_us":345,"threads_started":1}
I20260810 11:31:13.054212 16604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:13.041580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51318 (local address 127.8.18.194:45817)
0810 11:31:13.043971 (+ 2391us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:13.043981 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:13.044028 (+ 47us) server_negotiation.cc:407] Connection header received
0810 11:31:13.044132 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:13.044141 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:13.044408 (+ 267us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:13.045289 (+ 881us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:31:13.045550 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:13.047044 (+ 1494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:13.050034 (+ 2990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:13.051839 (+ 1805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:13.053324 (+ 1485us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:13.053640 (+ 316us) server_negotiation.cc:1035] Waiting for connection context
0810 11:31:13.053729 (+ 89us) server_negotiation.cc:299] Negotiation successful
0810 11:31:13.053924 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2207,"thread_start_us":2075,"threads_started":1}
I20260810 11:31:13.054472 16606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:13.043022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.195:35295 (local address 127.0.0.1:56726)
0810 11:31:13.044429 (+ 1407us) negotiation.cc:107] Waiting for socket to connect
0810 11:31:13.044454 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0810 11:31:13.044608 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:31:13.046106 (+ 1498us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:31:13.046111 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:31:13.046437 (+ 326us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:31:13.046899 (+ 462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:13.046914 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:13.048700 (+ 1786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:13.048708 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:13.050998 (+ 2290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:13.051016 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:13.053628 (+ 2612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:13.053635 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:13.053822 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:13.053858 (+ 36us) client_negotiation.cc:769] Sending connection context
0810 11:31:13.053962 (+ 104us) client_negotiation.cc:240] Negotiation successful
0810 11:31:13.054108 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1250,"mutex_wait_us":44,"thread_start_us":1099,"threads_started":1}
I20260810 11:31:13.054989 16605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:13.042534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56726 (local address 127.8.18.195:35295)
0810 11:31:13.045448 (+ 2914us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:13.045456 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:13.045494 (+ 38us) server_negotiation.cc:407] Connection header received
0810 11:31:13.045592 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:13.045603 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:13.045883 (+ 280us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:13.046094 (+ 211us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:31:13.047208 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:13.048543 (+ 1335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:13.051267 (+ 2724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:13.053299 (+ 2032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:13.053494 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:13.053744 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0810 11:31:13.054028 (+ 284us) server_negotiation.cc:299] Negotiation successful
0810 11:31:13.054233 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2712,"thread_start_us":2386,"threads_started":1}
I20260810 11:31:13.056578 16215 tablet_service.cc:1467] Processing CreateTablet for tablet b22d12fbc1ea4a5ca32a8bfa416282db (DEFAULT_TABLE table=test-table [id=ec6c74d23bba4515ba12a3c4ccfa3ee8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:31:13.058394 16215 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b22d12fbc1ea4a5ca32a8bfa416282db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:31:13.058218 16363 tablet_service.cc:1467] Processing CreateTablet for tablet b22d12fbc1ea4a5ca32a8bfa416282db (DEFAULT_TABLE table=test-table [id=ec6c74d23bba4515ba12a3c4ccfa3ee8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:31:13.058808 16510 tablet_service.cc:1467] Processing CreateTablet for tablet b22d12fbc1ea4a5ca32a8bfa416282db (DEFAULT_TABLE table=test-table [id=ec6c74d23bba4515ba12a3c4ccfa3ee8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:31:13.059890 16363 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b22d12fbc1ea4a5ca32a8bfa416282db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:31:13.060334 16510 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b22d12fbc1ea4a5ca32a8bfa416282db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:31:13.073776 16607 tablet_bootstrap.cc:492] T b22d12fbc1ea4a5ca32a8bfa416282db P 85786fb38f3f4634ae1b440c35d07d3a: Bootstrap starting.
I20260810 11:31:13.077090 16608 tablet_bootstrap.cc:492] T b22d12fbc1ea4a5ca32a8bfa416282db P 83fb5fdfb5e34ed399aad0e87934d93c: Bootstrap starting.
I20260810 11:31:13.077795 16607 tablet_bootstrap.cc:654] T b22d12fbc1ea4a5ca32a8bfa416282db P 85786fb38f3f4634ae1b440c35d07d3a: Neither blocks nor log segments found. Creating new log.
I20260810 11:31:13.078765 16609 tablet_bootstrap.cc:492] T b22d12fbc1ea4a5ca32a8bfa416282db P fc6dfd46f9cc4201886c620db1510312: Bootstrap starting.
I20260810 11:31:13.079818 16607 log.cc:824] T b22d12fbc1ea4a5ca32a8bfa416282db P 85786fb38f3f4634ae1b440c35d07d3a: Log is configured to *not* fsync() on all Append() calls
I20260810 11:31:13.081743 16608 tablet_bootstrap.cc:654] T b22d12fbc1ea4a5ca32a8bfa416282db P 83fb5fdfb5e34ed399aad0e87934d93c: Neither blocks nor log segments found. Creating new log.
I20260810 11:31:13.082604 16609 tablet_bootstrap.cc:654] T b22d12fbc1ea4a5ca32a8bfa416282db P fc6dfd46f9cc4201886c620db1510312: Neither blocks nor log segments found. Creating new log.
I20260810 11:31:13.083068 16608 log.cc:824] T b22d12fbc1ea4a5ca32a8bfa416282db P 83fb5fdfb5e34ed399aad0e87934d93c: Log is configured to *not* fsync() on all Append() calls
I20260810 11:31:13.083907 16609 log.cc:824] T b22d12fbc1ea4a5ca32a8bfa416282db P fc6dfd46f9cc4201886c620db1510312: Log is configured to *not* fsync() on all Append() calls
I20260810 11:31:13.084478 16607 tablet_bootstrap.cc:492] T b22d12fbc1ea4a5ca32a8bfa416282db P 85786fb38f3f4634ae1b440c35d07d3a: No bootstrap required, opened a new log
I20260810 11:31:13.084836 16607 ts_tablet_manager.cc:1397] T b22d12fbc1ea4a5ca32a8bfa416282db P 85786fb38f3f4634ae1b440c35d07d3a: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.004s
I20260810 11:31:13.086308 16608 tablet_bootstrap.cc:492] T b22d12fbc1ea4a5ca32a8bfa416282db P 83fb5fdfb5e34ed399aad0e87934d93c: No bootstrap required, opened a new log
I20260810 11:31:13.086609 16608 ts_tablet_manager.cc:1397] T b22d12fbc1ea4a5ca32a8bfa416282db P 83fb5fdfb5e34ed399aad0e87934d93c: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.004s
I20260810 11:31:13.087174 16609 tablet_bootstrap.cc:492] T b22d12fbc1ea4a5ca32a8bfa416282db P fc6dfd46f9cc4201886c620db1510312: No bootstrap required, opened a new log
I20260810 11:31:13.087481 16609 ts_tablet_manager.cc:1397] T b22d12fbc1ea4a5ca32a8bfa416282db P fc6dfd46f9cc4201886c620db1510312: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260810 11:31:13.095268 16608 raft_consensus.cc:348] T b22d12fbc1ea4a5ca32a8bfa416282db P 83fb5fdfb5e34ed399aad0e87934d93c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85786fb38f3f4634ae1b440c35d07d3a" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 41221 } } peers { permanent_uuid: "83fb5fdfb5e34ed399aad0e87934d93c" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 45817 } } peers { permanent_uuid: "fc6dfd46f9cc4201886c620db1510312" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 35295 } }
I20260810 11:31:13.095753 16608 raft_consensus.cc:374] T b22d12fbc1ea4a5ca32a8bfa416282db P 83fb5fdfb5e34ed399aad0e87934d93c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:31:13.095860 16608 raft_consensus.cc:729] T b22d12fbc1ea4a5ca32a8bfa416282db P 83fb5fdfb5e34ed399aad0e87934d93c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83fb5fdfb5e34ed399aad0e87934d93c, State: Initialized, Role: FOLLOWER
I20260810 11:31:13.096565 16608 consensus_queue.cc:260] T b22d12fbc1ea4a5ca32a8bfa416282db P 83fb5fdfb5e34ed399aad0e87934d93c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85786fb38f3f4634ae1b440c35d07d3a" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 41221 } } peers { permanent_uuid: "83fb5fdfb5e34ed399aad0e87934d93c" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 45817 } } peers { permanent_uuid: "fc6dfd46f9cc4201886c620db1510312" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 35295 } }
I20260810 11:31:13.096522 16607 raft_consensus.cc:348] T b22d12fbc1ea4a5ca32a8bfa416282db P 85786fb38f3f4634ae1b440c35d07d3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85786fb38f3f4634ae1b440c35d07d3a" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 41221 } } peers { permanent_uuid: "83fb5fdfb5e34ed399aad0e87934d93c" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 45817 } } peers { permanent_uuid: "fc6dfd46f9cc4201886c620db1510312" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 35295 } }
I20260810 11:31:13.097018 16607 raft_consensus.cc:374] T b22d12fbc1ea4a5ca32a8bfa416282db P 85786fb38f3f4634ae1b440c35d07d3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:31:13.097127 16607 raft_consensus.cc:729] T b22d12fbc1ea4a5ca32a8bfa416282db P 85786fb38f3f4634ae1b440c35d07d3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85786fb38f3f4634ae1b440c35d07d3a, State: Initialized, Role: FOLLOWER
I20260810 11:31:13.098050 16436 heartbeater.cc:503] Master 127.8.18.254:38029 was elected leader, sending a full tablet report...
I20260810 11:31:13.097883 16607 consensus_queue.cc:260] T b22d12fbc1ea4a5ca32a8bfa416282db P 85786fb38f3f4634ae1b440c35d07d3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85786fb38f3f4634ae1b440c35d07d3a" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 41221 } } peers { permanent_uuid: "83fb5fdfb5e34ed399aad0e87934d93c" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 45817 } } peers { permanent_uuid: "fc6dfd46f9cc4201886c620db1510312" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 35295 } }
I20260810 11:31:13.098784 16608 ts_tablet_manager.cc:1428] T b22d12fbc1ea4a5ca32a8bfa416282db P 83fb5fdfb5e34ed399aad0e87934d93c: Time spent starting tablet: real 0.012s user 0.012s sys 0.000s
I20260810 11:31:13.099752 16607 ts_tablet_manager.cc:1428] T b22d12fbc1ea4a5ca32a8bfa416282db P 85786fb38f3f4634ae1b440c35d07d3a: Time spent starting tablet: real 0.015s user 0.016s sys 0.000s
I20260810 11:31:13.100843 16609 raft_consensus.cc:348] T b22d12fbc1ea4a5ca32a8bfa416282db P fc6dfd46f9cc4201886c620db1510312 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85786fb38f3f4634ae1b440c35d07d3a" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 41221 } } peers { permanent_uuid: "83fb5fdfb5e34ed399aad0e87934d93c" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 45817 } } peers { permanent_uuid: "fc6dfd46f9cc4201886c620db1510312" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 35295 } }
I20260810 11:31:13.101369 16609 raft_consensus.cc:374] T b22d12fbc1ea4a5ca32a8bfa416282db P fc6dfd46f9cc4201886c620db1510312 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:31:13.101512 16609 raft_consensus.cc:729] T b22d12fbc1ea4a5ca32a8bfa416282db P fc6dfd46f9cc4201886c620db1510312 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc6dfd46f9cc4201886c620db1510312, State: Initialized, Role: FOLLOWER
I20260810 11:31:13.102138 16609 consensus_queue.cc:260] T b22d12fbc1ea4a5ca32a8bfa416282db P fc6dfd46f9cc4201886c620db1510312 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85786fb38f3f4634ae1b440c35d07d3a" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 41221 } } peers { permanent_uuid: "83fb5fdfb5e34ed399aad0e87934d93c" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 45817 } } peers { permanent_uuid: "fc6dfd46f9cc4201886c620db1510312" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 35295 } }
W20260810 11:31:13.103259 16437 tablet.cc:2369] T b22d12fbc1ea4a5ca32a8bfa416282db P 83fb5fdfb5e34ed399aad0e87934d93c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 11:31:13.104789 16581 heartbeater.cc:503] Master 127.8.18.254:38029 was elected leader, sending a full tablet report...
I20260810 11:31:13.105556 16609 ts_tablet_manager.cc:1428] T b22d12fbc1ea4a5ca32a8bfa416282db P fc6dfd46f9cc4201886c620db1510312: Time spent starting tablet: real 0.018s user 0.014s sys 0.004s
W20260810 11:31:13.172741 16582 tablet.cc:2369] T b22d12fbc1ea4a5ca32a8bfa416282db P fc6dfd46f9cc4201886c620db1510312: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 11:31:13.230387 16614 raft_consensus.cc:482] T b22d12fbc1ea4a5ca32a8bfa416282db P 83fb5fdfb5e34ed399aad0e87934d93c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 11:31:13.230793 16614 raft_consensus.cc:504] T b22d12fbc1ea4a5ca32a8bfa416282db P 83fb5fdfb5e34ed399aad0e87934d93c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85786fb38f3f4634ae1b440c35d07d3a" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 41221 } } peers { permanent_uuid: "83fb5fdfb5e34ed399aad0e87934d93c" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 45817 } } peers { permanent_uuid: "fc6dfd46f9cc4201886c620db1510312" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 35295 } }
I20260810 11:31:13.232867 16614 leader_election.cc:288] T b22d12fbc1ea4a5ca32a8bfa416282db P 83fb5fdfb5e34ed399aad0e87934d93c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 85786fb38f3f4634ae1b440c35d07d3a (127.8.18.193:41221), fc6dfd46f9cc4201886c620db1510312 (127.8.18.195:35295)
W20260810 11:31:13.234925 16291 tablet.cc:2369] T b22d12fbc1ea4a5ca32a8bfa416282db P 85786fb38f3f4634ae1b440c35d07d3a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 11:31:13.240455 16602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:13.233486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.194:55279 (local address 127.8.18.193:41221)
0810 11:31:13.233777 (+ 291us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:13.233787 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:13.234258 (+ 471us) server_negotiation.cc:407] Connection header received
0810 11:31:13.234629 (+ 371us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:13.234635 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:13.234720 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:13.234790 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:31:13.235890 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:13.236824 (+ 934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:13.238695 (+ 1871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:13.239478 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:13.239674 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:13.239766 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0810 11:31:13.240047 (+ 281us) server_negotiation.cc:299] Negotiation successful
0810 11:31:13.240196 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260810 11:31:13.240592 16617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:13.233294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.193:41221 (local address 127.8.18.194:55279)
0810 11:31:13.234090 (+ 796us) negotiation.cc:107] Waiting for socket to connect
0810 11:31:13.234117 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0810 11:31:13.234407 (+ 290us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:31:13.235258 (+ 851us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:31:13.235262 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:31:13.235293 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:31:13.235701 (+ 408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:13.235715 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:13.236948 (+ 1233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:13.236956 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:13.238525 (+ 1569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:13.238538 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:13.239712 (+ 1174us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:13.239718 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:13.239879 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:13.239904 (+ 25us) client_negotiation.cc:769] Sending connection context
0810 11:31:13.240017 (+ 113us) client_negotiation.cc:240] Negotiation successful
0810 11:31:13.240166 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":658,"mutex_wait_us":24,"thread_start_us":493,"threads_started":1}
I20260810 11:31:13.241961 16605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:13.234608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.194:33319 (local address 127.8.18.195:35295)
0810 11:31:13.235100 (+ 492us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:13.235110 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:13.235347 (+ 237us) server_negotiation.cc:407] Connection header received
0810 11:31:13.235677 (+ 330us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:13.235683 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:13.235765 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:13.235840 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:31:13.236830 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:13.237682 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:13.239622 (+ 1940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:13.240508 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:13.240769 (+ 261us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:13.240842 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0810 11:31:13.241504 (+ 662us) server_negotiation.cc:299] Negotiation successful
0810 11:31:13.241669 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":331}
I20260810 11:31:13.242220 16618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:13.234306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.195:35295 (local address 127.8.18.194:33319)
0810 11:31:13.235213 (+ 907us) negotiation.cc:107] Waiting for socket to connect
0810 11:31:13.235235 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 11:31:13.235450 (+ 215us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:31:13.236064 (+ 614us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:31:13.236069 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:31:13.236095 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:31:13.236609 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:13.236625 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:13.237834 (+ 1209us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:13.237842 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:13.239429 (+ 1587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:13.239442 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:13.241042 (+ 1600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:13.241052 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:13.241250 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:13.241299 (+ 49us) client_negotiation.cc:769] Sending connection context
0810 11:31:13.241459 (+ 160us) client_negotiation.cc:240] Negotiation successful
0810 11:31:13.241663 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":768,"mutex_wait_us":1,"spinlock_wait_cycles":10624,"thread_start_us":561,"threads_started":1}
I20260810 11:31:13.242010 16236 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b22d12fbc1ea4a5ca32a8bfa416282db" candidate_uuid: "83fb5fdfb5e34ed399aad0e87934d93c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85786fb38f3f4634ae1b440c35d07d3a" is_pre_election: true
I20260810 11:31:13.242656 16236 raft_consensus.cc:2454] T b22d12fbc1ea4a5ca32a8bfa416282db P 85786fb38f3f4634ae1b440c35d07d3a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83fb5fdfb5e34ed399aad0e87934d93c in term 0.
I20260810 11:31:13.243355 16312 leader_election.cc:302] T b22d12fbc1ea4a5ca32a8bfa416282db P 83fb5fdfb5e34ed399aad0e87934d93c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83fb5fdfb5e34ed399aad0e87934d93c, 85786fb38f3f4634ae1b440c35d07d3a; no voters:
I20260810 11:31:13.243423 16530 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b22d12fbc1ea4a5ca32a8bfa416282db" candidate_uuid: "83fb5fdfb5e34ed399aad0e87934d93c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc6dfd46f9cc4201886c620db1510312" is_pre_election: true
I20260810 11:31:13.243785 16614 raft_consensus.cc:2781] T b22d12fbc1ea4a5ca32a8bfa416282db P 83fb5fdfb5e34ed399aad0e87934d93c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 11:31:13.243965 16614 raft_consensus.cc:482] T b22d12fbc1ea4a5ca32a8bfa416282db P 83fb5fdfb5e34ed399aad0e87934d93c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 11:31:13.244117 16614 raft_consensus.cc:3037] T b22d12fbc1ea4a5ca32a8bfa416282db P 83fb5fdfb5e34ed399aad0e87934d93c [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:31:13.244107 16530 raft_consensus.cc:2454] T b22d12fbc1ea4a5ca32a8bfa416282db P fc6dfd46f9cc4201886c620db1510312 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83fb5fdfb5e34ed399aad0e87934d93c in term 0.
I20260810 11:31:13.246842 16614 raft_consensus.cc:504] T b22d12fbc1ea4a5ca32a8bfa416282db P 83fb5fdfb5e34ed399aad0e87934d93c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85786fb38f3f4634ae1b440c35d07d3a" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 41221 } } peers { permanent_uuid: "83fb5fdfb5e34ed399aad0e87934d93c" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 45817 } } peers { permanent_uuid: "fc6dfd46f9cc4201886c620db1510312" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 35295 } }
I20260810 11:31:13.247512 16614 leader_election.cc:288] T b22d12fbc1ea4a5ca32a8bfa416282db P 83fb5fdfb5e34ed399aad0e87934d93c [CANDIDATE]: Term 1 election: Requested vote from peers 85786fb38f3f4634ae1b440c35d07d3a (127.8.18.193:41221), fc6dfd46f9cc4201886c620db1510312 (127.8.18.195:35295)
I20260810 11:31:13.248273 16236 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b22d12fbc1ea4a5ca32a8bfa416282db" candidate_uuid: "83fb5fdfb5e34ed399aad0e87934d93c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85786fb38f3f4634ae1b440c35d07d3a"
I20260810 11:31:13.248302 16530 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b22d12fbc1ea4a5ca32a8bfa416282db" candidate_uuid: "83fb5fdfb5e34ed399aad0e87934d93c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc6dfd46f9cc4201886c620db1510312"
I20260810 11:31:13.248519 16236 raft_consensus.cc:3037] T b22d12fbc1ea4a5ca32a8bfa416282db P 85786fb38f3f4634ae1b440c35d07d3a [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:31:13.248608 16530 raft_consensus.cc:3037] T b22d12fbc1ea4a5ca32a8bfa416282db P fc6dfd46f9cc4201886c620db1510312 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:31:13.251466 16236 raft_consensus.cc:2454] T b22d12fbc1ea4a5ca32a8bfa416282db P 85786fb38f3f4634ae1b440c35d07d3a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83fb5fdfb5e34ed399aad0e87934d93c in term 1.
I20260810 11:31:13.251475 16530 raft_consensus.cc:2454] T b22d12fbc1ea4a5ca32a8bfa416282db P fc6dfd46f9cc4201886c620db1510312 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83fb5fdfb5e34ed399aad0e87934d93c in term 1.
I20260810 11:31:13.252074 16312 leader_election.cc:302] T b22d12fbc1ea4a5ca32a8bfa416282db P 83fb5fdfb5e34ed399aad0e87934d93c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83fb5fdfb5e34ed399aad0e87934d93c, fc6dfd46f9cc4201886c620db1510312; no voters:
I20260810 11:31:13.252439 16614 raft_consensus.cc:2781] T b22d12fbc1ea4a5ca32a8bfa416282db P 83fb5fdfb5e34ed399aad0e87934d93c [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:31:13.252951 16614 raft_consensus.cc:686] T b22d12fbc1ea4a5ca32a8bfa416282db P 83fb5fdfb5e34ed399aad0e87934d93c [term 1 LEADER]: Becoming Leader. State: Replica: 83fb5fdfb5e34ed399aad0e87934d93c, State: Running, Role: LEADER
I20260810 11:31:13.253536 16614 consensus_queue.cc:237] T b22d12fbc1ea4a5ca32a8bfa416282db P 83fb5fdfb5e34ed399aad0e87934d93c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85786fb38f3f4634ae1b440c35d07d3a" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 41221 } } peers { permanent_uuid: "83fb5fdfb5e34ed399aad0e87934d93c" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 45817 } } peers { permanent_uuid: "fc6dfd46f9cc4201886c620db1510312" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 35295 } }
I20260810 11:31:13.259219 16073 catalog_manager.cc:5495] T b22d12fbc1ea4a5ca32a8bfa416282db P 83fb5fdfb5e34ed399aad0e87934d93c reported cstate change: term changed from 0 to 1, leader changed from <none> to 83fb5fdfb5e34ed399aad0e87934d93c (127.8.18.194). New cstate: current_term: 1 leader_uuid: "83fb5fdfb5e34ed399aad0e87934d93c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85786fb38f3f4634ae1b440c35d07d3a" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 41221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "83fb5fdfb5e34ed399aad0e87934d93c" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 45817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc6dfd46f9cc4201886c620db1510312" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 35295 } health_report { overall_health: UNKNOWN } } }
I20260810 11:31:13.297168 16604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:13.291194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51322 (local address 127.8.18.194:45817)
0810 11:31:13.291423 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:13.291428 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:13.291461 (+ 33us) server_negotiation.cc:407] Connection header received
0810 11:31:13.291653 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:13.291657 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:13.291757 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:13.291899 (+ 142us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 11:31:13.292387 (+ 488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:13.293128 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:13.294731 (+ 1603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:13.295173 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:13.295306 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:13.295765 (+ 459us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 11:31:13.296470 (+ 705us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 11:31:13.296596 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0810 11:31:13.296743 (+ 147us) server_negotiation.cc:299] Negotiation successful
0810 11:31:13.296824 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260810 11:31:13.306700 16530 raft_consensus.cc:1261] T b22d12fbc1ea4a5ca32a8bfa416282db P fc6dfd46f9cc4201886c620db1510312 [term 1 FOLLOWER]: Refusing update from remote peer 83fb5fdfb5e34ed399aad0e87934d93c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 11:31:13.306692 16236 raft_consensus.cc:1261] T b22d12fbc1ea4a5ca32a8bfa416282db P 85786fb38f3f4634ae1b440c35d07d3a [term 1 FOLLOWER]: Refusing update from remote peer 83fb5fdfb5e34ed399aad0e87934d93c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 11:31:13.307771 16619 consensus_queue.cc:1035] T b22d12fbc1ea4a5ca32a8bfa416282db P 83fb5fdfb5e34ed399aad0e87934d93c [LEADER]: Connected to new peer: Peer: permanent_uuid: "85786fb38f3f4634ae1b440c35d07d3a" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 41221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 11:31:13.308172 16614 consensus_queue.cc:1035] T b22d12fbc1ea4a5ca32a8bfa416282db P 83fb5fdfb5e34ed399aad0e87934d93c [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc6dfd46f9cc4201886c620db1510312" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 35295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 11:31:13.343497 16073 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50930:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 11:31:13.343847 16073 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50930:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 11:31:13.350733 16073 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 064ed84ccdb040029c1070bb3cbfe22c: Sending DeleteTablet for 3 replicas of tablet b22d12fbc1ea4a5ca32a8bfa416282db
I20260810 11:31:13.352449 16510 tablet_service.cc:1514] Processing DeleteTablet for tablet b22d12fbc1ea4a5ca32a8bfa416282db with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:31:13 UTC) from {username='oryx', principal='oryx/127.8.18.254@KRBTEST.COM'} at 127.0.0.1:56726
I20260810 11:31:13.352679 16363 tablet_service.cc:1514] Processing DeleteTablet for tablet b22d12fbc1ea4a5ca32a8bfa416282db with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:31:13 UTC) from {username='oryx', principal='oryx/127.8.18.254@KRBTEST.COM'} at 127.0.0.1:51318
I20260810 11:31:13.353657 16215 tablet_service.cc:1514] Processing DeleteTablet for tablet b22d12fbc1ea4a5ca32a8bfa416282db with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:31:13 UTC) from {username='oryx', principal='oryx/127.8.18.254@KRBTEST.COM'} at 127.0.0.1:36068
I20260810 11:31:13.353891 16075 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50930:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 11:31:13.354791 16075 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 11:31:13.365989 16214 tablet_service.cc:1467] Processing CreateTablet for tablet d460000ceb954f87ba2c73ff0157ce30 (DEFAULT_TABLE table=test-table [id=76e4a7b9d03541cf8daee8bc5568a92b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:31:13.366614 16214 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d460000ceb954f87ba2c73ff0157ce30. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:31:13.368745 16633 tablet_replica.cc:330] T b22d12fbc1ea4a5ca32a8bfa416282db P 83fb5fdfb5e34ed399aad0e87934d93c: stopping tablet replica
I20260810 11:31:13.371189 16509 tablet_service.cc:1467] Processing CreateTablet for tablet d460000ceb954f87ba2c73ff0157ce30 (DEFAULT_TABLE table=test-table [id=76e4a7b9d03541cf8daee8bc5568a92b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:31:13.371472 16632 tablet_replica.cc:330] T b22d12fbc1ea4a5ca32a8bfa416282db P fc6dfd46f9cc4201886c620db1510312: stopping tablet replica
I20260810 11:31:13.371526 16634 tablet_replica.cc:330] T b22d12fbc1ea4a5ca32a8bfa416282db P 85786fb38f3f4634ae1b440c35d07d3a: stopping tablet replica
I20260810 11:31:13.371806 16509 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d460000ceb954f87ba2c73ff0157ce30. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:31:13.371991 16634 raft_consensus.cc:2229] T b22d12fbc1ea4a5ca32a8bfa416282db P 85786fb38f3f4634ae1b440c35d07d3a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 11:31:13.371994 16632 raft_consensus.cc:2229] T b22d12fbc1ea4a5ca32a8bfa416282db P fc6dfd46f9cc4201886c620db1510312 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 11:31:13.372614 16634 raft_consensus.cc:2258] T b22d12fbc1ea4a5ca32a8bfa416282db P 85786fb38f3f4634ae1b440c35d07d3a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:31:13.373385 16632 raft_consensus.cc:2258] T b22d12fbc1ea4a5ca32a8bfa416282db P fc6dfd46f9cc4201886c620db1510312 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:31:13.375646 16634 ts_tablet_manager.cc:1905] T b22d12fbc1ea4a5ca32a8bfa416282db P 85786fb38f3f4634ae1b440c35d07d3a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:31:13.376041 16632 ts_tablet_manager.cc:1905] T b22d12fbc1ea4a5ca32a8bfa416282db P fc6dfd46f9cc4201886c620db1510312: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:31:13.378901 16609 tablet_bootstrap.cc:492] T d460000ceb954f87ba2c73ff0157ce30 P fc6dfd46f9cc4201886c620db1510312: Bootstrap starting.
I20260810 11:31:13.380740 16609 tablet_bootstrap.cc:654] T d460000ceb954f87ba2c73ff0157ce30 P fc6dfd46f9cc4201886c620db1510312: Neither blocks nor log segments found. Creating new log.
I20260810 11:31:13.382903 16607 tablet_bootstrap.cc:492] T d460000ceb954f87ba2c73ff0157ce30 P 85786fb38f3f4634ae1b440c35d07d3a: Bootstrap starting.
I20260810 11:31:13.369319 16633 raft_consensus.cc:2229] T b22d12fbc1ea4a5ca32a8bfa416282db P 83fb5fdfb5e34ed399aad0e87934d93c [term 1 LEADER]: Raft consensus shutting down.
I20260810 11:31:13.383934 16362 tablet_service.cc:1467] Processing CreateTablet for tablet d460000ceb954f87ba2c73ff0157ce30 (DEFAULT_TABLE table=test-table [id=76e4a7b9d03541cf8daee8bc5568a92b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:31:13.384665 16633 raft_consensus.cc:2258] T b22d12fbc1ea4a5ca32a8bfa416282db P 83fb5fdfb5e34ed399aad0e87934d93c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:31:13.384825 16607 tablet_bootstrap.cc:654] T d460000ceb954f87ba2c73ff0157ce30 P 85786fb38f3f4634ae1b440c35d07d3a: Neither blocks nor log segments found. Creating new log.
I20260810 11:31:13.385592 16362 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d460000ceb954f87ba2c73ff0157ce30. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:31:13.385684 16634 ts_tablet_manager.cc:1918] T b22d12fbc1ea4a5ca32a8bfa416282db P 85786fb38f3f4634ae1b440c35d07d3a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 11:31:13.385886 16634 log.cc:1196] T b22d12fbc1ea4a5ca32a8bfa416282db P 85786fb38f3f4634ae1b440c35d07d3a: Deleting WAL directory at /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786361401922362-8267-0/minicluster-data/ts-0/wal/wals/b22d12fbc1ea4a5ca32a8bfa416282db
I20260810 11:31:13.386600 16634 ts_tablet_manager.cc:1939] T b22d12fbc1ea4a5ca32a8bfa416282db P 85786fb38f3f4634ae1b440c35d07d3a: Deleting consensus metadata
I20260810 11:31:13.390259 16060 catalog_manager.cc:4841] TS 85786fb38f3f4634ae1b440c35d07d3a (127.8.18.193:41221): tablet b22d12fbc1ea4a5ca32a8bfa416282db (table test-table [id=ec6c74d23bba4515ba12a3c4ccfa3ee8]) successfully deleted
I20260810 11:31:13.391907 16632 ts_tablet_manager.cc:1918] T b22d12fbc1ea4a5ca32a8bfa416282db P fc6dfd46f9cc4201886c620db1510312: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 11:31:13.392129 16632 log.cc:1196] T b22d12fbc1ea4a5ca32a8bfa416282db P fc6dfd46f9cc4201886c620db1510312: Deleting WAL directory at /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786361401922362-8267-0/minicluster-data/ts-2/wal/wals/b22d12fbc1ea4a5ca32a8bfa416282db
I20260810 11:31:13.392741 16632 ts_tablet_manager.cc:1939] T b22d12fbc1ea4a5ca32a8bfa416282db P fc6dfd46f9cc4201886c620db1510312: Deleting consensus metadata
I20260810 11:31:13.392932 16607 tablet_bootstrap.cc:492] T d460000ceb954f87ba2c73ff0157ce30 P 85786fb38f3f4634ae1b440c35d07d3a: No bootstrap required, opened a new log
I20260810 11:31:13.393163 16607 ts_tablet_manager.cc:1397] T d460000ceb954f87ba2c73ff0157ce30 P 85786fb38f3f4634ae1b440c35d07d3a: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.004s
I20260810 11:31:13.394191 16607 raft_consensus.cc:348] T d460000ceb954f87ba2c73ff0157ce30 P 85786fb38f3f4634ae1b440c35d07d3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85786fb38f3f4634ae1b440c35d07d3a" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 41221 } } peers { permanent_uuid: "83fb5fdfb5e34ed399aad0e87934d93c" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 45817 } } peers { permanent_uuid: "fc6dfd46f9cc4201886c620db1510312" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 35295 } }
I20260810 11:31:13.394444 16607 raft_consensus.cc:374] T d460000ceb954f87ba2c73ff0157ce30 P 85786fb38f3f4634ae1b440c35d07d3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:31:13.394548 16607 raft_consensus.cc:729] T d460000ceb954f87ba2c73ff0157ce30 P 85786fb38f3f4634ae1b440c35d07d3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85786fb38f3f4634ae1b440c35d07d3a, State: Initialized, Role: FOLLOWER
I20260810 11:31:13.394855 16607 consensus_queue.cc:260] T d460000ceb954f87ba2c73ff0157ce30 P 85786fb38f3f4634ae1b440c35d07d3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85786fb38f3f4634ae1b440c35d07d3a" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 41221 } } peers { permanent_uuid: "83fb5fdfb5e34ed399aad0e87934d93c" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 45817 } } peers { permanent_uuid: "fc6dfd46f9cc4201886c620db1510312" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 35295 } }
I20260810 11:31:13.395150 16060 catalog_manager.cc:4841] TS fc6dfd46f9cc4201886c620db1510312 (127.8.18.195:35295): tablet b22d12fbc1ea4a5ca32a8bfa416282db (table test-table [id=ec6c74d23bba4515ba12a3c4ccfa3ee8]) successfully deleted
I20260810 11:31:13.396678 16609 tablet_bootstrap.cc:492] T d460000ceb954f87ba2c73ff0157ce30 P fc6dfd46f9cc4201886c620db1510312: No bootstrap required, opened a new log
I20260810 11:31:13.396934 16609 ts_tablet_manager.cc:1397] T d460000ceb954f87ba2c73ff0157ce30 P fc6dfd46f9cc4201886c620db1510312: Time spent bootstrapping tablet: real 0.018s user 0.003s sys 0.007s
I20260810 11:31:13.398164 16607 ts_tablet_manager.cc:1428] T d460000ceb954f87ba2c73ff0157ce30 P 85786fb38f3f4634ae1b440c35d07d3a: Time spent starting tablet: real 0.005s user 0.000s sys 0.002s
I20260810 11:31:13.398168 16633 ts_tablet_manager.cc:1905] T b22d12fbc1ea4a5ca32a8bfa416282db P 83fb5fdfb5e34ed399aad0e87934d93c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:31:13.398469 16609 raft_consensus.cc:348] T d460000ceb954f87ba2c73ff0157ce30 P fc6dfd46f9cc4201886c620db1510312 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85786fb38f3f4634ae1b440c35d07d3a" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 41221 } } peers { permanent_uuid: "83fb5fdfb5e34ed399aad0e87934d93c" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 45817 } } peers { permanent_uuid: "fc6dfd46f9cc4201886c620db1510312" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 35295 } }
I20260810 11:31:13.399375 16609 raft_consensus.cc:374] T d460000ceb954f87ba2c73ff0157ce30 P fc6dfd46f9cc4201886c620db1510312 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:31:13.399597 16609 raft_consensus.cc:729] T d460000ceb954f87ba2c73ff0157ce30 P fc6dfd46f9cc4201886c620db1510312 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc6dfd46f9cc4201886c620db1510312, State: Initialized, Role: FOLLOWER
I20260810 11:31:13.399986 16609 consensus_queue.cc:260] T d460000ceb954f87ba2c73ff0157ce30 P fc6dfd46f9cc4201886c620db1510312 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85786fb38f3f4634ae1b440c35d07d3a" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 41221 } } peers { permanent_uuid: "83fb5fdfb5e34ed399aad0e87934d93c" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 45817 } } peers { permanent_uuid: "fc6dfd46f9cc4201886c620db1510312" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 35295 } }
I20260810 11:31:13.400741 16609 ts_tablet_manager.cc:1428] T d460000ceb954f87ba2c73ff0157ce30 P fc6dfd46f9cc4201886c620db1510312: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260810 11:31:13.405793 16608 tablet_bootstrap.cc:492] T d460000ceb954f87ba2c73ff0157ce30 P 83fb5fdfb5e34ed399aad0e87934d93c: Bootstrap starting.
I20260810 11:31:13.407830 16608 tablet_bootstrap.cc:654] T d460000ceb954f87ba2c73ff0157ce30 P 83fb5fdfb5e34ed399aad0e87934d93c: Neither blocks nor log segments found. Creating new log.
I20260810 11:31:13.409667 16633 ts_tablet_manager.cc:1918] T b22d12fbc1ea4a5ca32a8bfa416282db P 83fb5fdfb5e34ed399aad0e87934d93c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 11:31:13.409902 16633 log.cc:1196] T b22d12fbc1ea4a5ca32a8bfa416282db P 83fb5fdfb5e34ed399aad0e87934d93c: Deleting WAL directory at /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786361401922362-8267-0/minicluster-data/ts-1/wal/wals/b22d12fbc1ea4a5ca32a8bfa416282db
I20260810 11:31:13.410257 16608 tablet_bootstrap.cc:492] T d460000ceb954f87ba2c73ff0157ce30 P 83fb5fdfb5e34ed399aad0e87934d93c: No bootstrap required, opened a new log
I20260810 11:31:13.410454 16608 ts_tablet_manager.cc:1397] T d460000ceb954f87ba2c73ff0157ce30 P 83fb5fdfb5e34ed399aad0e87934d93c: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260810 11:31:13.410595 16633 ts_tablet_manager.cc:1939] T b22d12fbc1ea4a5ca32a8bfa416282db P 83fb5fdfb5e34ed399aad0e87934d93c: Deleting consensus metadata
I20260810 11:31:13.411336 16608 raft_consensus.cc:348] T d460000ceb954f87ba2c73ff0157ce30 P 83fb5fdfb5e34ed399aad0e87934d93c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85786fb38f3f4634ae1b440c35d07d3a" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 41221 } } peers { permanent_uuid: "83fb5fdfb5e34ed399aad0e87934d93c" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 45817 } } peers { permanent_uuid: "fc6dfd46f9cc4201886c620db1510312" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 35295 } }
I20260810 11:31:13.411564 16608 raft_consensus.cc:374] T d460000ceb954f87ba2c73ff0157ce30 P 83fb5fdfb5e34ed399aad0e87934d93c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:31:13.411674 16608 raft_consensus.cc:729] T d460000ceb954f87ba2c73ff0157ce30 P 83fb5fdfb5e34ed399aad0e87934d93c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83fb5fdfb5e34ed399aad0e87934d93c, State: Initialized, Role: FOLLOWER
I20260810 11:31:13.411994 16608 consensus_queue.cc:260] T d460000ceb954f87ba2c73ff0157ce30 P 83fb5fdfb5e34ed399aad0e87934d93c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85786fb38f3f4634ae1b440c35d07d3a" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 41221 } } peers { permanent_uuid: "83fb5fdfb5e34ed399aad0e87934d93c" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 45817 } } peers { permanent_uuid: "fc6dfd46f9cc4201886c620db1510312" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 35295 } }
I20260810 11:31:13.412864 16060 catalog_manager.cc:4841] TS 83fb5fdfb5e34ed399aad0e87934d93c (127.8.18.194:45817): tablet b22d12fbc1ea4a5ca32a8bfa416282db (table test-table [id=ec6c74d23bba4515ba12a3c4ccfa3ee8]) successfully deleted
I20260810 11:31:13.413276 16608 ts_tablet_manager.cc:1428] T d460000ceb954f87ba2c73ff0157ce30 P 83fb5fdfb5e34ed399aad0e87934d93c: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260810 11:31:13.422344 16614 raft_consensus.cc:482] T d460000ceb954f87ba2c73ff0157ce30 P 83fb5fdfb5e34ed399aad0e87934d93c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 11:31:13.422605 16614 raft_consensus.cc:504] T d460000ceb954f87ba2c73ff0157ce30 P 83fb5fdfb5e34ed399aad0e87934d93c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85786fb38f3f4634ae1b440c35d07d3a" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 41221 } } peers { permanent_uuid: "83fb5fdfb5e34ed399aad0e87934d93c" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 45817 } } peers { permanent_uuid: "fc6dfd46f9cc4201886c620db1510312" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 35295 } }
I20260810 11:31:13.423336 16614 leader_election.cc:288] T d460000ceb954f87ba2c73ff0157ce30 P 83fb5fdfb5e34ed399aad0e87934d93c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 85786fb38f3f4634ae1b440c35d07d3a (127.8.18.193:41221), fc6dfd46f9cc4201886c620db1510312 (127.8.18.195:35295)
I20260810 11:31:13.424078 16236 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d460000ceb954f87ba2c73ff0157ce30" candidate_uuid: "83fb5fdfb5e34ed399aad0e87934d93c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85786fb38f3f4634ae1b440c35d07d3a" is_pre_election: true
I20260810 11:31:13.424171 16530 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d460000ceb954f87ba2c73ff0157ce30" candidate_uuid: "83fb5fdfb5e34ed399aad0e87934d93c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc6dfd46f9cc4201886c620db1510312" is_pre_election: true
I20260810 11:31:13.424382 16236 raft_consensus.cc:2454] T d460000ceb954f87ba2c73ff0157ce30 P 85786fb38f3f4634ae1b440c35d07d3a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83fb5fdfb5e34ed399aad0e87934d93c in term 0.
I20260810 11:31:13.424397 16530 raft_consensus.cc:2454] T d460000ceb954f87ba2c73ff0157ce30 P fc6dfd46f9cc4201886c620db1510312 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83fb5fdfb5e34ed399aad0e87934d93c in term 0.
I20260810 11:31:13.424865 16312 leader_election.cc:302] T d460000ceb954f87ba2c73ff0157ce30 P 83fb5fdfb5e34ed399aad0e87934d93c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83fb5fdfb5e34ed399aad0e87934d93c, fc6dfd46f9cc4201886c620db1510312; no voters:
I20260810 11:31:13.425173 16614 raft_consensus.cc:2781] T d460000ceb954f87ba2c73ff0157ce30 P 83fb5fdfb5e34ed399aad0e87934d93c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 11:31:13.425328 16614 raft_consensus.cc:482] T d460000ceb954f87ba2c73ff0157ce30 P 83fb5fdfb5e34ed399aad0e87934d93c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 11:31:13.425460 16614 raft_consensus.cc:3037] T d460000ceb954f87ba2c73ff0157ce30 P 83fb5fdfb5e34ed399aad0e87934d93c [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:31:13.428890 16614 raft_consensus.cc:504] T d460000ceb954f87ba2c73ff0157ce30 P 83fb5fdfb5e34ed399aad0e87934d93c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85786fb38f3f4634ae1b440c35d07d3a" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 41221 } } peers { permanent_uuid: "83fb5fdfb5e34ed399aad0e87934d93c" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 45817 } } peers { permanent_uuid: "fc6dfd46f9cc4201886c620db1510312" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 35295 } }
I20260810 11:31:13.429782 16614 leader_election.cc:288] T d460000ceb954f87ba2c73ff0157ce30 P 83fb5fdfb5e34ed399aad0e87934d93c [CANDIDATE]: Term 1 election: Requested vote from peers 85786fb38f3f4634ae1b440c35d07d3a (127.8.18.193:41221), fc6dfd46f9cc4201886c620db1510312 (127.8.18.195:35295)
I20260810 11:31:13.430408 16236 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d460000ceb954f87ba2c73ff0157ce30" candidate_uuid: "83fb5fdfb5e34ed399aad0e87934d93c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85786fb38f3f4634ae1b440c35d07d3a"
I20260810 11:31:13.430425 16530 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d460000ceb954f87ba2c73ff0157ce30" candidate_uuid: "83fb5fdfb5e34ed399aad0e87934d93c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc6dfd46f9cc4201886c620db1510312"
I20260810 11:31:13.430656 16236 raft_consensus.cc:3037] T d460000ceb954f87ba2c73ff0157ce30 P 85786fb38f3f4634ae1b440c35d07d3a [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:31:13.430667 16530 raft_consensus.cc:3037] T d460000ceb954f87ba2c73ff0157ce30 P fc6dfd46f9cc4201886c620db1510312 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:31:13.433521 16236 raft_consensus.cc:2454] T d460000ceb954f87ba2c73ff0157ce30 P 85786fb38f3f4634ae1b440c35d07d3a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83fb5fdfb5e34ed399aad0e87934d93c in term 1.
I20260810 11:31:13.433933 16530 raft_consensus.cc:2454] T d460000ceb954f87ba2c73ff0157ce30 P fc6dfd46f9cc4201886c620db1510312 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83fb5fdfb5e34ed399aad0e87934d93c in term 1.
I20260810 11:31:13.434065 16312 leader_election.cc:302] T d460000ceb954f87ba2c73ff0157ce30 P 83fb5fdfb5e34ed399aad0e87934d93c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83fb5fdfb5e34ed399aad0e87934d93c, 85786fb38f3f4634ae1b440c35d07d3a; no voters:
I20260810 11:31:13.434442 16614 raft_consensus.cc:2781] T d460000ceb954f87ba2c73ff0157ce30 P 83fb5fdfb5e34ed399aad0e87934d93c [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:31:13.434710 16614 raft_consensus.cc:686] T d460000ceb954f87ba2c73ff0157ce30 P 83fb5fdfb5e34ed399aad0e87934d93c [term 1 LEADER]: Becoming Leader. State: Replica: 83fb5fdfb5e34ed399aad0e87934d93c, State: Running, Role: LEADER
I20260810 11:31:13.435032 16614 consensus_queue.cc:237] T d460000ceb954f87ba2c73ff0157ce30 P 83fb5fdfb5e34ed399aad0e87934d93c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85786fb38f3f4634ae1b440c35d07d3a" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 41221 } } peers { permanent_uuid: "83fb5fdfb5e34ed399aad0e87934d93c" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 45817 } } peers { permanent_uuid: "fc6dfd46f9cc4201886c620db1510312" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 35295 } }
I20260810 11:31:13.438330 16075 catalog_manager.cc:5495] T d460000ceb954f87ba2c73ff0157ce30 P 83fb5fdfb5e34ed399aad0e87934d93c reported cstate change: term changed from 0 to 1, leader changed from <none> to 83fb5fdfb5e34ed399aad0e87934d93c (127.8.18.194). New cstate: current_term: 1 leader_uuid: "83fb5fdfb5e34ed399aad0e87934d93c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85786fb38f3f4634ae1b440c35d07d3a" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 41221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "83fb5fdfb5e34ed399aad0e87934d93c" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 45817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc6dfd46f9cc4201886c620db1510312" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 35295 } health_report { overall_health: UNKNOWN } } }
Aug 10 11:31:13 dist-test-slave-srgv krb5kdc[16029](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361470, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.18.254@KRBTEST.COM for oryx/127.8.18.254@KRBTEST.COM
I20260810 11:31:13.497826 16644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:13.480295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:38029 (local address 127.0.0.1:50934)
0810 11:31:13.481115 (+ 820us) negotiation.cc:107] Waiting for socket to connect
0810 11:31:13.481149 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0810 11:31:13.481301 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:31:13.481724 (+ 423us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:31:13.481728 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:31:13.482475 (+ 747us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:31:13.482777 (+ 302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:13.482787 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:13.484363 (+ 1576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:13.484367 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:13.485544 (+ 1177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:13.485560 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:13.486660 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:13.486665 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:13.486795 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:13.487626 (+ 831us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:31:13.487647 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:31:13.490120 (+ 2473us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:31:13.494580 (+ 4460us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:31:13.494597 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:31:13.494605 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:31:13.495136 (+ 531us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:31:13.495664 (+ 528us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:31:13.495668 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:31:13.495671 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:31:13.495834 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:31:13.496523 (+ 689us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:31:13.496530 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:31:13.496697 (+ 167us) client_negotiation.cc:769] Sending connection context
0810 11:31:13.496911 (+ 214us) client_negotiation.cc:240] Negotiation successful
0810 11:31:13.497287 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":640,"thread_start_us":459,"threads_started":1}
I20260810 11:31:13.497972 16466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:13.480289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50934 (local address 127.8.18.254:38029)
0810 11:31:13.480585 (+ 296us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:13.480592 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:13.481264 (+ 672us) server_negotiation.cc:407] Connection header received
0810 11:31:13.481453 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:13.481461 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:13.481549 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:13.481685 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:31:13.482973 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:13.484192 (+ 1219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:13.485745 (+ 1553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:13.486427 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:13.486571 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:13.490370 (+ 3799us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:31:13.490380 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:31:13.490386 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:31:13.490433 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:31:13.494343 (+ 3910us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:31:13.495299 (+ 956us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:31:13.495307 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:31:13.495316 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:31:13.495513 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:31:13.495956 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:31:13.495962 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:31:13.495965 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:31:13.496355 (+ 390us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:31:13.496530 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0810 11:31:13.497035 (+ 505us) server_negotiation.cc:299] Negotiation successful
0810 11:31:13.497539 (+ 504us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":170,"server-negotiator.queue_time_us":137}
I20260810 11:31:13.504103 16075 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.8.18.254@KRBTEST.COM'} at 127.0.0.1:50934:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260810 11:31:13.505602 16075 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 11:31:13.513670 16362 tablet_service.cc:1467] Processing CreateTablet for tablet 21ff3b53b3f246b7a3bec64e673a4ab0 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=a23cc002d5614d67b3ca3e202eb96475]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 11:31:13.513648 16215 tablet_service.cc:1467] Processing CreateTablet for tablet 21ff3b53b3f246b7a3bec64e673a4ab0 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=a23cc002d5614d67b3ca3e202eb96475]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 11:31:13.514323 16362 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 21ff3b53b3f246b7a3bec64e673a4ab0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:31:13.514348 16215 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 21ff3b53b3f246b7a3bec64e673a4ab0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:31:13.518177 16509 tablet_service.cc:1467] Processing CreateTablet for tablet 21ff3b53b3f246b7a3bec64e673a4ab0 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=a23cc002d5614d67b3ca3e202eb96475]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 11:31:13.518817 16509 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 21ff3b53b3f246b7a3bec64e673a4ab0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:31:13.522361 16608 tablet_bootstrap.cc:492] T 21ff3b53b3f246b7a3bec64e673a4ab0 P 83fb5fdfb5e34ed399aad0e87934d93c: Bootstrap starting.
I20260810 11:31:13.524163 16608 tablet_bootstrap.cc:654] T 21ff3b53b3f246b7a3bec64e673a4ab0 P 83fb5fdfb5e34ed399aad0e87934d93c: Neither blocks nor log segments found. Creating new log.
I20260810 11:31:13.525722 16607 tablet_bootstrap.cc:492] T 21ff3b53b3f246b7a3bec64e673a4ab0 P 85786fb38f3f4634ae1b440c35d07d3a: Bootstrap starting.
I20260810 11:31:13.527324 16607 tablet_bootstrap.cc:654] T 21ff3b53b3f246b7a3bec64e673a4ab0 P 85786fb38f3f4634ae1b440c35d07d3a: Neither blocks nor log segments found. Creating new log.
I20260810 11:31:13.528342 16609 tablet_bootstrap.cc:492] T 21ff3b53b3f246b7a3bec64e673a4ab0 P fc6dfd46f9cc4201886c620db1510312: Bootstrap starting.
I20260810 11:31:13.528908 16608 tablet_bootstrap.cc:492] T 21ff3b53b3f246b7a3bec64e673a4ab0 P 83fb5fdfb5e34ed399aad0e87934d93c: No bootstrap required, opened a new log
I20260810 11:31:13.529088 16608 ts_tablet_manager.cc:1397] T 21ff3b53b3f246b7a3bec64e673a4ab0 P 83fb5fdfb5e34ed399aad0e87934d93c: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.000s
I20260810 11:31:13.529990 16608 raft_consensus.cc:348] T 21ff3b53b3f246b7a3bec64e673a4ab0 P 83fb5fdfb5e34ed399aad0e87934d93c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85786fb38f3f4634ae1b440c35d07d3a" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 41221 } } peers { permanent_uuid: "83fb5fdfb5e34ed399aad0e87934d93c" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 45817 } } peers { permanent_uuid: "fc6dfd46f9cc4201886c620db1510312" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 35295 } }
I20260810 11:31:13.530229 16609 tablet_bootstrap.cc:654] T 21ff3b53b3f246b7a3bec64e673a4ab0 P fc6dfd46f9cc4201886c620db1510312: Neither blocks nor log segments found. Creating new log.
I20260810 11:31:13.530303 16608 raft_consensus.cc:374] T 21ff3b53b3f246b7a3bec64e673a4ab0 P 83fb5fdfb5e34ed399aad0e87934d93c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:31:13.530396 16608 raft_consensus.cc:729] T 21ff3b53b3f246b7a3bec64e673a4ab0 P 83fb5fdfb5e34ed399aad0e87934d93c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83fb5fdfb5e34ed399aad0e87934d93c, State: Initialized, Role: FOLLOWER
I20260810 11:31:13.530791 16608 consensus_queue.cc:260] T 21ff3b53b3f246b7a3bec64e673a4ab0 P 83fb5fdfb5e34ed399aad0e87934d93c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85786fb38f3f4634ae1b440c35d07d3a" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 41221 } } peers { permanent_uuid: "83fb5fdfb5e34ed399aad0e87934d93c" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 45817 } } peers { permanent_uuid: "fc6dfd46f9cc4201886c620db1510312" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 35295 } }
I20260810 11:31:13.531332 16614 tablet_replica.cc:439] T 21ff3b53b3f246b7a3bec64e673a4ab0 P 83fb5fdfb5e34ed399aad0e87934d93c: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85786fb38f3f4634ae1b440c35d07d3a" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 41221 } } peers { permanent_uuid: "83fb5fdfb5e34ed399aad0e87934d93c" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 45817 } } peers { permanent_uuid: "fc6dfd46f9cc4201886c620db1510312" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 35295 } } }
I20260810 11:31:13.531790 16614 tablet_replica.cc:442] T 21ff3b53b3f246b7a3bec64e673a4ab0 P 83fb5fdfb5e34ed399aad0e87934d93c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 11:31:13.536226 16608 ts_tablet_manager.cc:1428] T 21ff3b53b3f246b7a3bec64e673a4ab0 P 83fb5fdfb5e34ed399aad0e87934d93c: Time spent starting tablet: real 0.007s user 0.003s sys 0.000s
I20260810 11:31:13.541671 16609 tablet_bootstrap.cc:492] T 21ff3b53b3f246b7a3bec64e673a4ab0 P fc6dfd46f9cc4201886c620db1510312: No bootstrap required, opened a new log
I20260810 11:31:13.541918 16609 ts_tablet_manager.cc:1397] T 21ff3b53b3f246b7a3bec64e673a4ab0 P fc6dfd46f9cc4201886c620db1510312: Time spent bootstrapping tablet: real 0.014s user 0.004s sys 0.006s
I20260810 11:31:13.541992 16607 tablet_bootstrap.cc:492] T 21ff3b53b3f246b7a3bec64e673a4ab0 P 85786fb38f3f4634ae1b440c35d07d3a: No bootstrap required, opened a new log
I20260810 11:31:13.542179 16607 ts_tablet_manager.cc:1397] T 21ff3b53b3f246b7a3bec64e673a4ab0 P 85786fb38f3f4634ae1b440c35d07d3a: Time spent bootstrapping tablet: real 0.017s user 0.009s sys 0.002s
I20260810 11:31:13.542834 16609 raft_consensus.cc:348] T 21ff3b53b3f246b7a3bec64e673a4ab0 P fc6dfd46f9cc4201886c620db1510312 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85786fb38f3f4634ae1b440c35d07d3a" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 41221 } } peers { permanent_uuid: "83fb5fdfb5e34ed399aad0e87934d93c" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 45817 } } peers { permanent_uuid: "fc6dfd46f9cc4201886c620db1510312" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 35295 } }
I20260810 11:31:13.542945 16607 raft_consensus.cc:348] T 21ff3b53b3f246b7a3bec64e673a4ab0 P 85786fb38f3f4634ae1b440c35d07d3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85786fb38f3f4634ae1b440c35d07d3a" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 41221 } } peers { permanent_uuid: "83fb5fdfb5e34ed399aad0e87934d93c" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 45817 } } peers { permanent_uuid: "fc6dfd46f9cc4201886c620db1510312" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 35295 } }
I20260810 11:31:13.543139 16609 raft_consensus.cc:374] T 21ff3b53b3f246b7a3bec64e673a4ab0 P fc6dfd46f9cc4201886c620db1510312 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:31:13.543200 16607 raft_consensus.cc:374] T 21ff3b53b3f246b7a3bec64e673a4ab0 P 85786fb38f3f4634ae1b440c35d07d3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:31:13.543236 16609 raft_consensus.cc:729] T 21ff3b53b3f246b7a3bec64e673a4ab0 P fc6dfd46f9cc4201886c620db1510312 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc6dfd46f9cc4201886c620db1510312, State: Initialized, Role: FOLLOWER
I20260810 11:31:13.543296 16607 raft_consensus.cc:729] T 21ff3b53b3f246b7a3bec64e673a4ab0 P 85786fb38f3f4634ae1b440c35d07d3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85786fb38f3f4634ae1b440c35d07d3a, State: Initialized, Role: FOLLOWER
I20260810 11:31:13.543641 16607 consensus_queue.cc:260] T 21ff3b53b3f246b7a3bec64e673a4ab0 P 85786fb38f3f4634ae1b440c35d07d3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85786fb38f3f4634ae1b440c35d07d3a" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 41221 } } peers { permanent_uuid: "83fb5fdfb5e34ed399aad0e87934d93c" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 45817 } } peers { permanent_uuid: "fc6dfd46f9cc4201886c620db1510312" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 35295 } }
I20260810 11:31:13.543817 16609 consensus_queue.cc:260] T 21ff3b53b3f246b7a3bec64e673a4ab0 P fc6dfd46f9cc4201886c620db1510312 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85786fb38f3f4634ae1b440c35d07d3a" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 41221 } } peers { permanent_uuid: "83fb5fdfb5e34ed399aad0e87934d93c" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 45817 } } peers { permanent_uuid: "fc6dfd46f9cc4201886c620db1510312" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 35295 } }
I20260810 11:31:13.544477 16609 ts_tablet_manager.cc:1428] T 21ff3b53b3f246b7a3bec64e673a4ab0 P fc6dfd46f9cc4201886c620db1510312: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260810 11:31:13.544196 16615 tablet_replica.cc:439] T 21ff3b53b3f246b7a3bec64e673a4ab0 P 85786fb38f3f4634ae1b440c35d07d3a: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85786fb38f3f4634ae1b440c35d07d3a" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 41221 } } peers { permanent_uuid: "83fb5fdfb5e34ed399aad0e87934d93c" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 45817 } } peers { permanent_uuid: "fc6dfd46f9cc4201886c620db1510312" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 35295 } } }
I20260810 11:31:13.544668 16615 tablet_replica.cc:442] T 21ff3b53b3f246b7a3bec64e673a4ab0 P 85786fb38f3f4634ae1b440c35d07d3a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 11:31:13.545656 16616 tablet_replica.cc:439] T 21ff3b53b3f246b7a3bec64e673a4ab0 P fc6dfd46f9cc4201886c620db1510312: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85786fb38f3f4634ae1b440c35d07d3a" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 41221 } } peers { permanent_uuid: "83fb5fdfb5e34ed399aad0e87934d93c" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 45817 } } peers { permanent_uuid: "fc6dfd46f9cc4201886c620db1510312" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 35295 } } }
I20260810 11:31:13.546029 16616 tablet_replica.cc:442] T 21ff3b53b3f246b7a3bec64e673a4ab0 P fc6dfd46f9cc4201886c620db1510312: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 11:31:13.548147 16607 ts_tablet_manager.cc:1428] T 21ff3b53b3f246b7a3bec64e673a4ab0 P 85786fb38f3f4634ae1b440c35d07d3a: Time spent starting tablet: real 0.006s user 0.003s sys 0.000s
I20260810 11:31:13.653632 16614 raft_consensus.cc:482] T 21ff3b53b3f246b7a3bec64e673a4ab0 P 83fb5fdfb5e34ed399aad0e87934d93c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 11:31:13.653914 16614 raft_consensus.cc:504] T 21ff3b53b3f246b7a3bec64e673a4ab0 P 83fb5fdfb5e34ed399aad0e87934d93c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85786fb38f3f4634ae1b440c35d07d3a" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 41221 } } peers { permanent_uuid: "83fb5fdfb5e34ed399aad0e87934d93c" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 45817 } } peers { permanent_uuid: "fc6dfd46f9cc4201886c620db1510312" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 35295 } }
I20260810 11:31:13.654738 16614 leader_election.cc:288] T 21ff3b53b3f246b7a3bec64e673a4ab0 P 83fb5fdfb5e34ed399aad0e87934d93c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 85786fb38f3f4634ae1b440c35d07d3a (127.8.18.193:41221), fc6dfd46f9cc4201886c620db1510312 (127.8.18.195:35295)
I20260810 11:31:13.655406 16236 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21ff3b53b3f246b7a3bec64e673a4ab0" candidate_uuid: "83fb5fdfb5e34ed399aad0e87934d93c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85786fb38f3f4634ae1b440c35d07d3a" is_pre_election: true
I20260810 11:31:13.655481 16530 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21ff3b53b3f246b7a3bec64e673a4ab0" candidate_uuid: "83fb5fdfb5e34ed399aad0e87934d93c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc6dfd46f9cc4201886c620db1510312" is_pre_election: true
I20260810 11:31:13.655752 16236 raft_consensus.cc:2454] T 21ff3b53b3f246b7a3bec64e673a4ab0 P 85786fb38f3f4634ae1b440c35d07d3a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83fb5fdfb5e34ed399aad0e87934d93c in term 0.
I20260810 11:31:13.655752 16530 raft_consensus.cc:2454] T 21ff3b53b3f246b7a3bec64e673a4ab0 P fc6dfd46f9cc4201886c620db1510312 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83fb5fdfb5e34ed399aad0e87934d93c in term 0.
I20260810 11:31:13.656270 16312 leader_election.cc:302] T 21ff3b53b3f246b7a3bec64e673a4ab0 P 83fb5fdfb5e34ed399aad0e87934d93c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83fb5fdfb5e34ed399aad0e87934d93c, fc6dfd46f9cc4201886c620db1510312; no voters:
I20260810 11:31:13.656674 16614 raft_consensus.cc:2781] T 21ff3b53b3f246b7a3bec64e673a4ab0 P 83fb5fdfb5e34ed399aad0e87934d93c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 11:31:13.656809 16614 raft_consensus.cc:482] T 21ff3b53b3f246b7a3bec64e673a4ab0 P 83fb5fdfb5e34ed399aad0e87934d93c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 11:31:13.656926 16614 raft_consensus.cc:3037] T 21ff3b53b3f246b7a3bec64e673a4ab0 P 83fb5fdfb5e34ed399aad0e87934d93c [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:31:13.660298 16614 raft_consensus.cc:504] T 21ff3b53b3f246b7a3bec64e673a4ab0 P 83fb5fdfb5e34ed399aad0e87934d93c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85786fb38f3f4634ae1b440c35d07d3a" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 41221 } } peers { permanent_uuid: "83fb5fdfb5e34ed399aad0e87934d93c" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 45817 } } peers { permanent_uuid: "fc6dfd46f9cc4201886c620db1510312" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 35295 } }
I20260810 11:31:13.661156 16614 leader_election.cc:288] T 21ff3b53b3f246b7a3bec64e673a4ab0 P 83fb5fdfb5e34ed399aad0e87934d93c [CANDIDATE]: Term 1 election: Requested vote from peers 85786fb38f3f4634ae1b440c35d07d3a (127.8.18.193:41221), fc6dfd46f9cc4201886c620db1510312 (127.8.18.195:35295)
I20260810 11:31:13.661975 16530 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21ff3b53b3f246b7a3bec64e673a4ab0" candidate_uuid: "83fb5fdfb5e34ed399aad0e87934d93c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc6dfd46f9cc4201886c620db1510312"
I20260810 11:31:13.661973 16236 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21ff3b53b3f246b7a3bec64e673a4ab0" candidate_uuid: "83fb5fdfb5e34ed399aad0e87934d93c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85786fb38f3f4634ae1b440c35d07d3a"
I20260810 11:31:13.662253 16530 raft_consensus.cc:3037] T 21ff3b53b3f246b7a3bec64e673a4ab0 P fc6dfd46f9cc4201886c620db1510312 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:31:13.662253 16236 raft_consensus.cc:3037] T 21ff3b53b3f246b7a3bec64e673a4ab0 P 85786fb38f3f4634ae1b440c35d07d3a [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:31:13.665477 16530 raft_consensus.cc:2454] T 21ff3b53b3f246b7a3bec64e673a4ab0 P fc6dfd46f9cc4201886c620db1510312 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83fb5fdfb5e34ed399aad0e87934d93c in term 1.
I20260810 11:31:13.665478 16236 raft_consensus.cc:2454] T 21ff3b53b3f246b7a3bec64e673a4ab0 P 85786fb38f3f4634ae1b440c35d07d3a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83fb5fdfb5e34ed399aad0e87934d93c in term 1.
I20260810 11:31:13.665992 16312 leader_election.cc:302] T 21ff3b53b3f246b7a3bec64e673a4ab0 P 83fb5fdfb5e34ed399aad0e87934d93c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83fb5fdfb5e34ed399aad0e87934d93c, fc6dfd46f9cc4201886c620db1510312; no voters:
I20260810 11:31:13.666344 16614 raft_consensus.cc:2781] T 21ff3b53b3f246b7a3bec64e673a4ab0 P 83fb5fdfb5e34ed399aad0e87934d93c [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:31:13.666754 16614 raft_consensus.cc:686] T 21ff3b53b3f246b7a3bec64e673a4ab0 P 83fb5fdfb5e34ed399aad0e87934d93c [term 1 LEADER]: Becoming Leader. State: Replica: 83fb5fdfb5e34ed399aad0e87934d93c, State: Running, Role: LEADER
I20260810 11:31:13.667075 16614 consensus_queue.cc:237] T 21ff3b53b3f246b7a3bec64e673a4ab0 P 83fb5fdfb5e34ed399aad0e87934d93c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85786fb38f3f4634ae1b440c35d07d3a" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 41221 } } peers { permanent_uuid: "83fb5fdfb5e34ed399aad0e87934d93c" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 45817 } } peers { permanent_uuid: "fc6dfd46f9cc4201886c620db1510312" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 35295 } }
I20260810 11:31:13.669162 16627 tablet_replica.cc:439] T 21ff3b53b3f246b7a3bec64e673a4ab0 P 83fb5fdfb5e34ed399aad0e87934d93c: TxnStatusTablet state changed. Reason: New leader 83fb5fdfb5e34ed399aad0e87934d93c. Latest consensus state: current_term: 1 leader_uuid: "83fb5fdfb5e34ed399aad0e87934d93c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85786fb38f3f4634ae1b440c35d07d3a" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 41221 } } peers { permanent_uuid: "83fb5fdfb5e34ed399aad0e87934d93c" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 45817 } } peers { permanent_uuid: "fc6dfd46f9cc4201886c620db1510312" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 35295 } } }
I20260810 11:31:13.669667 16627 tablet_replica.cc:442] T 21ff3b53b3f246b7a3bec64e673a4ab0 P 83fb5fdfb5e34ed399aad0e87934d93c: This TxnStatusTablet replica's current role is: LEADER
I20260810 11:31:13.670547 16649 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260810 11:31:13.670852 16649 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260810 11:31:13.670738 16075 catalog_manager.cc:5495] T 21ff3b53b3f246b7a3bec64e673a4ab0 P 83fb5fdfb5e34ed399aad0e87934d93c reported cstate change: term changed from 0 to 1, leader changed from <none> to 83fb5fdfb5e34ed399aad0e87934d93c (127.8.18.194). New cstate: current_term: 1 leader_uuid: "83fb5fdfb5e34ed399aad0e87934d93c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85786fb38f3f4634ae1b440c35d07d3a" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 41221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "83fb5fdfb5e34ed399aad0e87934d93c" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 45817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc6dfd46f9cc4201886c620db1510312" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 35295 } health_report { overall_health: UNKNOWN } } }
I20260810 11:31:13.702879 16644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:13.696517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.194:45817 (local address 127.0.0.1:51332)
0810 11:31:13.696844 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0810 11:31:13.696869 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0810 11:31:13.697010 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:31:13.697498 (+ 488us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:31:13.697502 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:31:13.697512 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 11:31:13.697702 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:13.697710 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:13.699266 (+ 1556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:13.699271 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:13.700504 (+ 1233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:13.700522 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:13.701648 (+ 1126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:13.701653 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:13.701800 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:13.701872 (+ 72us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 11:31:13.702383 (+ 511us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 11:31:13.702407 (+ 24us) client_negotiation.cc:769] Sending connection context
0810 11:31:13.702483 (+ 76us) client_negotiation.cc:240] Negotiation successful
0810 11:31:13.702632 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":66}
I20260810 11:31:13.702967 16604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:13.696765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51332 (local address 127.8.18.194:45817)
0810 11:31:13.696978 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:13.696984 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:13.697009 (+ 25us) server_negotiation.cc:407] Connection header received
0810 11:31:13.697199 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:13.697206 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:13.697303 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:13.697452 (+ 149us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 11:31:13.697847 (+ 395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:13.699074 (+ 1227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:13.700691 (+ 1617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:13.701462 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:13.701621 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:13.702075 (+ 454us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 11:31:13.702260 (+ 185us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 11:31:13.702372 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0810 11:31:13.702539 (+ 167us) server_negotiation.cc:299] Negotiation successful
0810 11:31:13.702647 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260810 11:31:13.707435 16236 raft_consensus.cc:1261] T 21ff3b53b3f246b7a3bec64e673a4ab0 P 85786fb38f3f4634ae1b440c35d07d3a [term 1 FOLLOWER]: Refusing update from remote peer 83fb5fdfb5e34ed399aad0e87934d93c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 11:31:13.707533 16530 raft_consensus.cc:1261] T 21ff3b53b3f246b7a3bec64e673a4ab0 P fc6dfd46f9cc4201886c620db1510312 [term 1 FOLLOWER]: Refusing update from remote peer 83fb5fdfb5e34ed399aad0e87934d93c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 11:31:13.708302 16614 consensus_queue.cc:1035] T 21ff3b53b3f246b7a3bec64e673a4ab0 P 83fb5fdfb5e34ed399aad0e87934d93c [LEADER]: Connected to new peer: Peer: permanent_uuid: "85786fb38f3f4634ae1b440c35d07d3a" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 41221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 11:31:13.708715 16627 consensus_queue.cc:1035] T 21ff3b53b3f246b7a3bec64e673a4ab0 P 83fb5fdfb5e34ed399aad0e87934d93c [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc6dfd46f9cc4201886c620db1510312" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 35295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 11:31:13.712085 16615 tablet_replica.cc:439] T 21ff3b53b3f246b7a3bec64e673a4ab0 P 85786fb38f3f4634ae1b440c35d07d3a: TxnStatusTablet state changed. Reason: New leader 83fb5fdfb5e34ed399aad0e87934d93c. Latest consensus state: current_term: 1 leader_uuid: "83fb5fdfb5e34ed399aad0e87934d93c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85786fb38f3f4634ae1b440c35d07d3a" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 41221 } } peers { permanent_uuid: "83fb5fdfb5e34ed399aad0e87934d93c" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 45817 } } peers { permanent_uuid: "fc6dfd46f9cc4201886c620db1510312" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 35295 } } }
I20260810 11:31:13.712416 16615 tablet_replica.cc:442] T 21ff3b53b3f246b7a3bec64e673a4ab0 P 85786fb38f3f4634ae1b440c35d07d3a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 11:31:13.715659 16622 mvcc.cc:205] Tried to move back new op lower bound from 7316936596298625024 to 7316936596143538176. Current Snapshot: MvccSnapshot[applied={T|T < 7316936596298625024}]
I20260810 11:31:13.717517 16627 tablet_replica.cc:439] T 21ff3b53b3f246b7a3bec64e673a4ab0 P 83fb5fdfb5e34ed399aad0e87934d93c: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "83fb5fdfb5e34ed399aad0e87934d93c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85786fb38f3f4634ae1b440c35d07d3a" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 41221 } } peers { permanent_uuid: "83fb5fdfb5e34ed399aad0e87934d93c" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 45817 } } peers { permanent_uuid: "fc6dfd46f9cc4201886c620db1510312" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 35295 } } }
I20260810 11:31:13.717819 16627 tablet_replica.cc:442] T 21ff3b53b3f246b7a3bec64e673a4ab0 P 83fb5fdfb5e34ed399aad0e87934d93c: This TxnStatusTablet replica's current role is: LEADER
I20260810 11:31:13.721853 16615 tablet_replica.cc:439] T 21ff3b53b3f246b7a3bec64e673a4ab0 P 85786fb38f3f4634ae1b440c35d07d3a: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "83fb5fdfb5e34ed399aad0e87934d93c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85786fb38f3f4634ae1b440c35d07d3a" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 41221 } } peers { permanent_uuid: "83fb5fdfb5e34ed399aad0e87934d93c" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 45817 } } peers { permanent_uuid: "fc6dfd46f9cc4201886c620db1510312" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 35295 } } }
I20260810 11:31:13.722146 16615 tablet_replica.cc:442] T 21ff3b53b3f246b7a3bec64e673a4ab0 P 85786fb38f3f4634ae1b440c35d07d3a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 11:31:13.725003 16616 tablet_replica.cc:439] T 21ff3b53b3f246b7a3bec64e673a4ab0 P fc6dfd46f9cc4201886c620db1510312: TxnStatusTablet state changed. Reason: New leader 83fb5fdfb5e34ed399aad0e87934d93c. Latest consensus state: current_term: 1 leader_uuid: "83fb5fdfb5e34ed399aad0e87934d93c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85786fb38f3f4634ae1b440c35d07d3a" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 41221 } } peers { permanent_uuid: "83fb5fdfb5e34ed399aad0e87934d93c" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 45817 } } peers { permanent_uuid: "fc6dfd46f9cc4201886c620db1510312" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 35295 } } }
I20260810 11:31:13.725311 16616 tablet_replica.cc:442] T 21ff3b53b3f246b7a3bec64e673a4ab0 P fc6dfd46f9cc4201886c620db1510312: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 11:31:13.726859 16619 tablet_replica.cc:439] T 21ff3b53b3f246b7a3bec64e673a4ab0 P 83fb5fdfb5e34ed399aad0e87934d93c: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "83fb5fdfb5e34ed399aad0e87934d93c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85786fb38f3f4634ae1b440c35d07d3a" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 41221 } } peers { permanent_uuid: "83fb5fdfb5e34ed399aad0e87934d93c" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 45817 } } peers { permanent_uuid: "fc6dfd46f9cc4201886c620db1510312" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 35295 } } }
I20260810 11:31:13.727249 16619 tablet_replica.cc:442] T 21ff3b53b3f246b7a3bec64e673a4ab0 P 83fb5fdfb5e34ed399aad0e87934d93c: This TxnStatusTablet replica's current role is: LEADER
I20260810 11:31:13.728662 16616 tablet_replica.cc:439] T 21ff3b53b3f246b7a3bec64e673a4ab0 P fc6dfd46f9cc4201886c620db1510312: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "83fb5fdfb5e34ed399aad0e87934d93c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85786fb38f3f4634ae1b440c35d07d3a" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 41221 } } peers { permanent_uuid: "83fb5fdfb5e34ed399aad0e87934d93c" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 45817 } } peers { permanent_uuid: "fc6dfd46f9cc4201886c620db1510312" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 35295 } } }
I20260810 11:31:13.728912 16616 tablet_replica.cc:442] T 21ff3b53b3f246b7a3bec64e673a4ab0 P fc6dfd46f9cc4201886c620db1510312: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 11:31:13.747207 16604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:13.739116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.194:55297 (local address 127.8.18.194:45817)
0810 11:31:13.739349 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:13.739356 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:13.740155 (+ 799us) server_negotiation.cc:407] Connection header received
0810 11:31:13.740300 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:13.740305 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:13.740401 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:13.740517 (+ 116us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 11:31:13.741327 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:13.742494 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:13.744657 (+ 2163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:13.745458 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:13.745625 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:13.746279 (+ 654us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 11:31:13.746435 (+ 156us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 11:31:13.746540 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0810 11:31:13.746730 (+ 190us) server_negotiation.cc:299] Negotiation successful
0810 11:31:13.746841 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":27,"server-negotiator.queue_time_us":67}
I20260810 11:31:13.747644 16655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:13.739226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.194:45817 (local address 127.8.18.194:55297)
0810 11:31:13.740038 (+ 812us) negotiation.cc:107] Waiting for socket to connect
0810 11:31:13.740054 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 11:31:13.740194 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:31:13.740522 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:31:13.740525 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:31:13.740535 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 11:31:13.741140 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:13.741153 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:13.742652 (+ 1499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:13.742658 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:13.744444 (+ 1786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:13.744462 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:13.745657 (+ 1195us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:13.745663 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:13.745904 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:13.745994 (+ 90us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 11:31:13.746564 (+ 570us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 11:31:13.746577 (+ 13us) client_negotiation.cc:769] Sending connection context
0810 11:31:13.746671 (+ 94us) client_negotiation.cc:240] Negotiation successful
0810 11:31:13.746853 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":617,"mutex_wait_us":4,"thread_start_us":464,"threads_started":1}
I20260810 11:31:13.760717 16236 raft_consensus.cc:1261] T d460000ceb954f87ba2c73ff0157ce30 P 85786fb38f3f4634ae1b440c35d07d3a [term 1 FOLLOWER]: Refusing update from remote peer 83fb5fdfb5e34ed399aad0e87934d93c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 11:31:13.761556 16530 raft_consensus.cc:1261] T d460000ceb954f87ba2c73ff0157ce30 P fc6dfd46f9cc4201886c620db1510312 [term 1 FOLLOWER]: Refusing update from remote peer 83fb5fdfb5e34ed399aad0e87934d93c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 11:31:13.762130 16614 consensus_queue.cc:1035] T d460000ceb954f87ba2c73ff0157ce30 P 83fb5fdfb5e34ed399aad0e87934d93c [LEADER]: Connected to new peer: Peer: permanent_uuid: "85786fb38f3f4634ae1b440c35d07d3a" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 41221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 11:31:13.762724 16614 consensus_queue.cc:1035] T d460000ceb954f87ba2c73ff0157ce30 P 83fb5fdfb5e34ed399aad0e87934d93c [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc6dfd46f9cc4201886c620db1510312" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 35295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 11:31:13.775956 16624 mvcc.cc:205] Tried to move back new op lower bound from 7316936596515913728 to 7316936595192627200. Current Snapshot: MvccSnapshot[applied={T|T < 7316936596515913728}]
I20260810 11:31:13.781917 16623 mvcc.cc:205] Tried to move back new op lower bound from 7316936596515913728 to 7316936595192627200. Current Snapshot: MvccSnapshot[applied={T|T < 7316936596515913728}]
I20260810 11:31:13.836165 16075 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50930:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 11:31:13.836483 16075 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50930:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 11:31:13.840237 16075 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 064ed84ccdb040029c1070bb3cbfe22c: Sending DeleteTablet for 3 replicas of tablet d460000ceb954f87ba2c73ff0157ce30
I20260810 11:31:13.841313 16215 tablet_service.cc:1514] Processing DeleteTablet for tablet d460000ceb954f87ba2c73ff0157ce30 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:31:13 UTC) from {username='oryx', principal='oryx/127.8.18.254@KRBTEST.COM'} at 127.0.0.1:36068
I20260810 11:31:13.841490 16362 tablet_service.cc:1514] Processing DeleteTablet for tablet d460000ceb954f87ba2c73ff0157ce30 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:31:13 UTC) from {username='oryx', principal='oryx/127.8.18.254@KRBTEST.COM'} at 127.0.0.1:51318
I20260810 11:31:13.841539 16509 tablet_service.cc:1514] Processing DeleteTablet for tablet d460000ceb954f87ba2c73ff0157ce30 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:31:13 UTC) from {username='oryx', principal='oryx/127.8.18.254@KRBTEST.COM'} at 127.0.0.1:56726
I20260810 11:31:13.841691 16634 tablet_replica.cc:330] T d460000ceb954f87ba2c73ff0157ce30 P 85786fb38f3f4634ae1b440c35d07d3a: stopping tablet replica
I20260810 11:31:13.841809 16633 tablet_replica.cc:330] T d460000ceb954f87ba2c73ff0157ce30 P 83fb5fdfb5e34ed399aad0e87934d93c: stopping tablet replica
I20260810 11:31:13.841856 16632 tablet_replica.cc:330] T d460000ceb954f87ba2c73ff0157ce30 P fc6dfd46f9cc4201886c620db1510312: stopping tablet replica
I20260810 11:31:13.841915 16634 raft_consensus.cc:2229] T d460000ceb954f87ba2c73ff0157ce30 P 85786fb38f3f4634ae1b440c35d07d3a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 11:31:13.842013 16633 raft_consensus.cc:2229] T d460000ceb954f87ba2c73ff0157ce30 P 83fb5fdfb5e34ed399aad0e87934d93c [term 1 LEADER]: Raft consensus shutting down.
I20260810 11:31:13.842059 16632 raft_consensus.cc:2229] T d460000ceb954f87ba2c73ff0157ce30 P fc6dfd46f9cc4201886c620db1510312 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 11:31:13.842128 16634 raft_consensus.cc:2258] T d460000ceb954f87ba2c73ff0157ce30 P 85786fb38f3f4634ae1b440c35d07d3a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:31:13.842257 16632 raft_consensus.cc:2258] T d460000ceb954f87ba2c73ff0157ce30 P fc6dfd46f9cc4201886c620db1510312 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:31:13.842371 16633 raft_consensus.cc:2258] T d460000ceb954f87ba2c73ff0157ce30 P 83fb5fdfb5e34ed399aad0e87934d93c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:31:13.843676 16632 ts_tablet_manager.cc:1905] T d460000ceb954f87ba2c73ff0157ce30 P fc6dfd46f9cc4201886c620db1510312: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:31:13.844254 16633 ts_tablet_manager.cc:1905] T d460000ceb954f87ba2c73ff0157ce30 P 83fb5fdfb5e34ed399aad0e87934d93c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:31:13.850700 16634 ts_tablet_manager.cc:1905] T d460000ceb954f87ba2c73ff0157ce30 P 85786fb38f3f4634ae1b440c35d07d3a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:31:13.852133 16632 ts_tablet_manager.cc:1918] T d460000ceb954f87ba2c73ff0157ce30 P fc6dfd46f9cc4201886c620db1510312: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 11:31:13.852321 16632 log.cc:1196] T d460000ceb954f87ba2c73ff0157ce30 P fc6dfd46f9cc4201886c620db1510312: Deleting WAL directory at /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786361401922362-8267-0/minicluster-data/ts-2/wal/wals/d460000ceb954f87ba2c73ff0157ce30
I20260810 11:31:13.852778 16632 ts_tablet_manager.cc:1939] T d460000ceb954f87ba2c73ff0157ce30 P fc6dfd46f9cc4201886c620db1510312: Deleting consensus metadata
I20260810 11:31:13.854045 16060 catalog_manager.cc:4841] TS fc6dfd46f9cc4201886c620db1510312 (127.8.18.195:35295): tablet d460000ceb954f87ba2c73ff0157ce30 (table test-table [id=76e4a7b9d03541cf8daee8bc5568a92b]) successfully deleted
I20260810 11:31:13.858997 16634 ts_tablet_manager.cc:1918] T d460000ceb954f87ba2c73ff0157ce30 P 85786fb38f3f4634ae1b440c35d07d3a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 11:31:13.859287 16634 log.cc:1196] T d460000ceb954f87ba2c73ff0157ce30 P 85786fb38f3f4634ae1b440c35d07d3a: Deleting WAL directory at /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786361401922362-8267-0/minicluster-data/ts-0/wal/wals/d460000ceb954f87ba2c73ff0157ce30
I20260810 11:31:13.859807 16634 ts_tablet_manager.cc:1939] T d460000ceb954f87ba2c73ff0157ce30 P 85786fb38f3f4634ae1b440c35d07d3a: Deleting consensus metadata
I20260810 11:31:13.861253 16060 catalog_manager.cc:4841] TS 85786fb38f3f4634ae1b440c35d07d3a (127.8.18.193:41221): tablet d460000ceb954f87ba2c73ff0157ce30 (table test-table [id=76e4a7b9d03541cf8daee8bc5568a92b]) successfully deleted
I20260810 11:31:13.861374 16633 ts_tablet_manager.cc:1918] T d460000ceb954f87ba2c73ff0157ce30 P 83fb5fdfb5e34ed399aad0e87934d93c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 11:31:13.861739 16633 log.cc:1196] T d460000ceb954f87ba2c73ff0157ce30 P 83fb5fdfb5e34ed399aad0e87934d93c: Deleting WAL directory at /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786361401922362-8267-0/minicluster-data/ts-1/wal/wals/d460000ceb954f87ba2c73ff0157ce30
I20260810 11:31:13.862270 16633 ts_tablet_manager.cc:1939] T d460000ceb954f87ba2c73ff0157ce30 P 83fb5fdfb5e34ed399aad0e87934d93c: Deleting consensus metadata
I20260810 11:31:13.863816 16060 catalog_manager.cc:4841] TS 83fb5fdfb5e34ed399aad0e87934d93c (127.8.18.194:45817): tablet d460000ceb954f87ba2c73ff0157ce30 (table test-table [id=76e4a7b9d03541cf8daee8bc5568a92b]) successfully deleted
Aug 10 11:31:13 dist-test-slave-srgv krb5kdc[16029](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361473, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 10 11:31:13 dist-test-slave-srgv krb5kdc[16029](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.8.18.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 11:31:13 dist-test-slave-srgv krb5kdc[16029](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.8.18.254@KRBTEST.COM, Server not found in Kerberos database
W20260810 11:31:13.909852 16466 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:31:13.897838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50948 (local address 127.8.18.254:38029)
0810 11:31:13.898070 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:13.898077 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:13.898469 (+ 392us) server_negotiation.cc:407] Connection header received
0810 11:31:13.898633 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:13.898641 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:13.898743 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:13.898871 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:31:13.900369 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:13.901672 (+ 1303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:13.903019 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:13.903709 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:13.903841 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:13.909573 (+ 5732us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50948: BlockingRecv error: recv got EOF from 127.0.0.1:50948 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
Aug 10 11:31:13 dist-test-slave-srgv krb5kdc[16029](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361473, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.8.18.254@KRBTEST.COM
I20260810 11:31:13.932878 16466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:13.913383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50958 (local address 127.8.18.254:38029)
0810 11:31:13.913648 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:13.913655 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:13.914036 (+ 381us) server_negotiation.cc:407] Connection header received
0810 11:31:13.914238 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:13.914242 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:13.914333 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:13.914424 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:31:13.916130 (+ 1706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:13.916875 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:13.918848 (+ 1973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:13.919540 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:13.919729 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:13.925125 (+ 5396us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:31:13.925135 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:31:13.925139 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:31:13.925194 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:31:13.929180 (+ 3986us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:31:13.930370 (+ 1190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:31:13.930378 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:31:13.930382 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:31:13.930583 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:31:13.931041 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:31:13.931048 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:31:13.931053 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:31:13.931453 (+ 400us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:31:13.931765 (+ 312us) server_negotiation.cc:1035] Waiting for connection context
0810 11:31:13.932082 (+ 317us) server_negotiation.cc:299] Negotiation successful
0810 11:31:13.932361 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104}
I20260810 11:31:13.937877 16075 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50958:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 11:31:13.938685 16075 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 11:31:13.952527 16215 tablet_service.cc:1467] Processing CreateTablet for tablet 300d9788081b429faac41e293d8f37dc (DEFAULT_TABLE table=test-table [id=e54a35124c094b9484438e06bcffb374]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:31:13.952656 16509 tablet_service.cc:1467] Processing CreateTablet for tablet 300d9788081b429faac41e293d8f37dc (DEFAULT_TABLE table=test-table [id=e54a35124c094b9484438e06bcffb374]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:31:13.953076 16215 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 300d9788081b429faac41e293d8f37dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:31:13.953143 16509 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 300d9788081b429faac41e293d8f37dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:31:13.957237 16362 tablet_service.cc:1467] Processing CreateTablet for tablet 300d9788081b429faac41e293d8f37dc (DEFAULT_TABLE table=test-table [id=e54a35124c094b9484438e06bcffb374]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:31:13.958005 16362 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 300d9788081b429faac41e293d8f37dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:31:13.960242 16607 tablet_bootstrap.cc:492] T 300d9788081b429faac41e293d8f37dc P 85786fb38f3f4634ae1b440c35d07d3a: Bootstrap starting.
I20260810 11:31:13.960245 16609 tablet_bootstrap.cc:492] T 300d9788081b429faac41e293d8f37dc P fc6dfd46f9cc4201886c620db1510312: Bootstrap starting.
I20260810 11:31:13.962064 16609 tablet_bootstrap.cc:654] T 300d9788081b429faac41e293d8f37dc P fc6dfd46f9cc4201886c620db1510312: Neither blocks nor log segments found. Creating new log.
I20260810 11:31:13.962062 16607 tablet_bootstrap.cc:654] T 300d9788081b429faac41e293d8f37dc P 85786fb38f3f4634ae1b440c35d07d3a: Neither blocks nor log segments found. Creating new log.
I20260810 11:31:13.969352 16608 tablet_bootstrap.cc:492] T 300d9788081b429faac41e293d8f37dc P 83fb5fdfb5e34ed399aad0e87934d93c: Bootstrap starting.
I20260810 11:31:13.971861 16608 tablet_bootstrap.cc:654] T 300d9788081b429faac41e293d8f37dc P 83fb5fdfb5e34ed399aad0e87934d93c: Neither blocks nor log segments found. Creating new log.
I20260810 11:31:13.971978 16607 tablet_bootstrap.cc:492] T 300d9788081b429faac41e293d8f37dc P 85786fb38f3f4634ae1b440c35d07d3a: No bootstrap required, opened a new log
I20260810 11:31:13.972168 16607 ts_tablet_manager.cc:1397] T 300d9788081b429faac41e293d8f37dc P 85786fb38f3f4634ae1b440c35d07d3a: Time spent bootstrapping tablet: real 0.012s user 0.010s sys 0.000s
I20260810 11:31:13.973050 16607 raft_consensus.cc:348] T 300d9788081b429faac41e293d8f37dc P 85786fb38f3f4634ae1b440c35d07d3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85786fb38f3f4634ae1b440c35d07d3a" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 41221 } } peers { permanent_uuid: "83fb5fdfb5e34ed399aad0e87934d93c" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 45817 } } peers { permanent_uuid: "fc6dfd46f9cc4201886c620db1510312" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 35295 } }
I20260810 11:31:13.973344 16607 raft_consensus.cc:374] T 300d9788081b429faac41e293d8f37dc P 85786fb38f3f4634ae1b440c35d07d3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:31:13.973465 16607 raft_consensus.cc:729] T 300d9788081b429faac41e293d8f37dc P 85786fb38f3f4634ae1b440c35d07d3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85786fb38f3f4634ae1b440c35d07d3a, State: Initialized, Role: FOLLOWER
I20260810 11:31:13.973738 16607 consensus_queue.cc:260] T 300d9788081b429faac41e293d8f37dc P 85786fb38f3f4634ae1b440c35d07d3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85786fb38f3f4634ae1b440c35d07d3a" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 41221 } } peers { permanent_uuid: "83fb5fdfb5e34ed399aad0e87934d93c" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 45817 } } peers { permanent_uuid: "fc6dfd46f9cc4201886c620db1510312" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 35295 } }
I20260810 11:31:13.976590 16609 tablet_bootstrap.cc:492] T 300d9788081b429faac41e293d8f37dc P fc6dfd46f9cc4201886c620db1510312: No bootstrap required, opened a new log
I20260810 11:31:13.976891 16609 ts_tablet_manager.cc:1397] T 300d9788081b429faac41e293d8f37dc P fc6dfd46f9cc4201886c620db1510312: Time spent bootstrapping tablet: real 0.017s user 0.007s sys 0.006s
I20260810 11:31:13.977046 16607 ts_tablet_manager.cc:1428] T 300d9788081b429faac41e293d8f37dc P 85786fb38f3f4634ae1b440c35d07d3a: Time spent starting tablet: real 0.005s user 0.003s sys 0.000s
I20260810 11:31:13.978245 16609 raft_consensus.cc:348] T 300d9788081b429faac41e293d8f37dc P fc6dfd46f9cc4201886c620db1510312 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85786fb38f3f4634ae1b440c35d07d3a" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 41221 } } peers { permanent_uuid: "83fb5fdfb5e34ed399aad0e87934d93c" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 45817 } } peers { permanent_uuid: "fc6dfd46f9cc4201886c620db1510312" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 35295 } }
I20260810 11:31:13.978744 16609 raft_consensus.cc:374] T 300d9788081b429faac41e293d8f37dc P fc6dfd46f9cc4201886c620db1510312 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:31:13.978915 16609 raft_consensus.cc:729] T 300d9788081b429faac41e293d8f37dc P fc6dfd46f9cc4201886c620db1510312 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc6dfd46f9cc4201886c620db1510312, State: Initialized, Role: FOLLOWER
I20260810 11:31:13.979260 16609 consensus_queue.cc:260] T 300d9788081b429faac41e293d8f37dc P fc6dfd46f9cc4201886c620db1510312 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85786fb38f3f4634ae1b440c35d07d3a" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 41221 } } peers { permanent_uuid: "83fb5fdfb5e34ed399aad0e87934d93c" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 45817 } } peers { permanent_uuid: "fc6dfd46f9cc4201886c620db1510312" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 35295 } }
I20260810 11:31:13.979959 16609 ts_tablet_manager.cc:1428] T 300d9788081b429faac41e293d8f37dc P fc6dfd46f9cc4201886c620db1510312: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260810 11:31:13.982707 16608 tablet_bootstrap.cc:492] T 300d9788081b429faac41e293d8f37dc P 83fb5fdfb5e34ed399aad0e87934d93c: No bootstrap required, opened a new log
I20260810 11:31:13.982914 16608 ts_tablet_manager.cc:1397] T 300d9788081b429faac41e293d8f37dc P 83fb5fdfb5e34ed399aad0e87934d93c: Time spent bootstrapping tablet: real 0.014s user 0.006s sys 0.003s
I20260810 11:31:13.983935 16608 raft_consensus.cc:348] T 300d9788081b429faac41e293d8f37dc P 83fb5fdfb5e34ed399aad0e87934d93c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85786fb38f3f4634ae1b440c35d07d3a" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 41221 } } peers { permanent_uuid: "83fb5fdfb5e34ed399aad0e87934d93c" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 45817 } } peers { permanent_uuid: "fc6dfd46f9cc4201886c620db1510312" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 35295 } }
I20260810 11:31:13.984315 16608 raft_consensus.cc:374] T 300d9788081b429faac41e293d8f37dc P 83fb5fdfb5e34ed399aad0e87934d93c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:31:13.984794 16608 raft_consensus.cc:729] T 300d9788081b429faac41e293d8f37dc P 83fb5fdfb5e34ed399aad0e87934d93c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83fb5fdfb5e34ed399aad0e87934d93c, State: Initialized, Role: FOLLOWER
I20260810 11:31:13.985119 16608 consensus_queue.cc:260] T 300d9788081b429faac41e293d8f37dc P 83fb5fdfb5e34ed399aad0e87934d93c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85786fb38f3f4634ae1b440c35d07d3a" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 41221 } } peers { permanent_uuid: "83fb5fdfb5e34ed399aad0e87934d93c" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 45817 } } peers { permanent_uuid: "fc6dfd46f9cc4201886c620db1510312" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 35295 } }
I20260810 11:31:13.986059 16608 ts_tablet_manager.cc:1428] T 300d9788081b429faac41e293d8f37dc P 83fb5fdfb5e34ed399aad0e87934d93c: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 11:31:14.008878 16616 raft_consensus.cc:482] T 300d9788081b429faac41e293d8f37dc P fc6dfd46f9cc4201886c620db1510312 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 11:31:14.009169 16616 raft_consensus.cc:504] T 300d9788081b429faac41e293d8f37dc P fc6dfd46f9cc4201886c620db1510312 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85786fb38f3f4634ae1b440c35d07d3a" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 41221 } } peers { permanent_uuid: "83fb5fdfb5e34ed399aad0e87934d93c" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 45817 } } peers { permanent_uuid: "fc6dfd46f9cc4201886c620db1510312" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 35295 } }
I20260810 11:31:14.011194 16616 leader_election.cc:288] T 300d9788081b429faac41e293d8f37dc P fc6dfd46f9cc4201886c620db1510312 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 85786fb38f3f4634ae1b440c35d07d3a (127.8.18.193:41221), 83fb5fdfb5e34ed399aad0e87934d93c (127.8.18.194:45817)
I20260810 11:31:14.035965 16677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:14.011678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.195:42561 (local address 127.8.18.193:41221)
0810 11:31:14.026813 (+ 15135us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:14.026820 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:14.026842 (+ 22us) server_negotiation.cc:407] Connection header received
0810 11:31:14.026921 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:14.026926 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:14.026998 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:14.027116 (+ 118us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:31:14.028763 (+ 1647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:14.030095 (+ 1332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:14.033195 (+ 3100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:14.034565 (+ 1370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:14.034742 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:14.034820 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0810 11:31:14.035411 (+ 591us) server_negotiation.cc:299] Negotiation successful
0810 11:31:14.035594 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14987,"thread_start_us":15562,"threads_started":1}
I20260810 11:31:14.036840 16676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:14.011476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.193:41221 (local address 127.8.18.195:42561)
0810 11:31:14.026108 (+ 14632us) negotiation.cc:107] Waiting for socket to connect
0810 11:31:14.026139 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0810 11:31:14.026350 (+ 211us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:31:14.028194 (+ 1844us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:31:14.028199 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:31:14.028220 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:31:14.028494 (+ 274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:14.028505 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:14.030308 (+ 1803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:14.030314 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:14.032924 (+ 2610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:14.032943 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:14.035011 (+ 2068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:14.035017 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:14.035209 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:14.035245 (+ 36us) client_negotiation.cc:769] Sending connection context
0810 11:31:14.036279 (+ 1034us) client_negotiation.cc:240] Negotiation successful
0810 11:31:14.036441 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14398,"mutex_wait_us":43,"thread_start_us":14183,"threads_started":1}
I20260810 11:31:14.037591 16236 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "300d9788081b429faac41e293d8f37dc" candidate_uuid: "fc6dfd46f9cc4201886c620db1510312" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85786fb38f3f4634ae1b440c35d07d3a" is_pre_election: true
I20260810 11:31:14.037874 16236 raft_consensus.cc:2454] T 300d9788081b429faac41e293d8f37dc P 85786fb38f3f4634ae1b440c35d07d3a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc6dfd46f9cc4201886c620db1510312 in term 0.
I20260810 11:31:14.038091 16604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:14.026639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.195:41029 (local address 127.8.18.194:45817)
0810 11:31:14.026835 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:14.026841 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:14.028010 (+ 1169us) server_negotiation.cc:407] Connection header received
0810 11:31:14.028190 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:14.028197 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:14.028286 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:14.028391 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:31:14.029237 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:14.030412 (+ 1175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:14.035036 (+ 4624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:14.036494 (+ 1458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:14.036635 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:14.036721 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0810 11:31:14.037645 (+ 924us) server_negotiation.cc:299] Negotiation successful
0810 11:31:14.037801 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260810 11:31:14.038465 16459 leader_election.cc:302] T 300d9788081b429faac41e293d8f37dc P fc6dfd46f9cc4201886c620db1510312 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85786fb38f3f4634ae1b440c35d07d3a, fc6dfd46f9cc4201886c620db1510312; no voters:
I20260810 11:31:14.038918 16616 raft_consensus.cc:2781] T 300d9788081b429faac41e293d8f37dc P fc6dfd46f9cc4201886c620db1510312 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 11:31:14.038968 16678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:14.027035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.194:45817 (local address 127.8.18.195:41029)
0810 11:31:14.027861 (+ 826us) negotiation.cc:107] Waiting for socket to connect
0810 11:31:14.027884 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0810 11:31:14.028037 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:31:14.028576 (+ 539us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:31:14.028581 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:31:14.028611 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:31:14.028993 (+ 382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:14.029006 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:14.030632 (+ 1626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:14.030637 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:14.034787 (+ 4150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:14.034807 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:14.037129 (+ 2322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:14.037208 (+ 79us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:14.037402 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:14.037466 (+ 64us) client_negotiation.cc:769] Sending connection context
0810 11:31:14.038364 (+ 898us) client_negotiation.cc:240] Negotiation successful
0810 11:31:14.038534 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":690,"mutex_wait_us":4,"thread_start_us":421,"threads_started":1}
I20260810 11:31:14.039047 16616 raft_consensus.cc:482] T 300d9788081b429faac41e293d8f37dc P fc6dfd46f9cc4201886c620db1510312 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 11:31:14.039155 16616 raft_consensus.cc:3037] T 300d9788081b429faac41e293d8f37dc P fc6dfd46f9cc4201886c620db1510312 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:31:14.042100 16384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "300d9788081b429faac41e293d8f37dc" candidate_uuid: "fc6dfd46f9cc4201886c620db1510312" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83fb5fdfb5e34ed399aad0e87934d93c" is_pre_election: true
I20260810 11:31:14.042423 16616 raft_consensus.cc:504] T 300d9788081b429faac41e293d8f37dc P fc6dfd46f9cc4201886c620db1510312 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85786fb38f3f4634ae1b440c35d07d3a" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 41221 } } peers { permanent_uuid: "83fb5fdfb5e34ed399aad0e87934d93c" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 45817 } } peers { permanent_uuid: "fc6dfd46f9cc4201886c620db1510312" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 35295 } }
I20260810 11:31:14.042661 16384 raft_consensus.cc:2454] T 300d9788081b429faac41e293d8f37dc P 83fb5fdfb5e34ed399aad0e87934d93c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc6dfd46f9cc4201886c620db1510312 in term 0.
I20260810 11:31:14.043100 16616 leader_election.cc:288] T 300d9788081b429faac41e293d8f37dc P fc6dfd46f9cc4201886c620db1510312 [CANDIDATE]: Term 1 election: Requested vote from peers 85786fb38f3f4634ae1b440c35d07d3a (127.8.18.193:41221), 83fb5fdfb5e34ed399aad0e87934d93c (127.8.18.194:45817)
I20260810 11:31:14.043951 16384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "300d9788081b429faac41e293d8f37dc" candidate_uuid: "fc6dfd46f9cc4201886c620db1510312" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83fb5fdfb5e34ed399aad0e87934d93c"
I20260810 11:31:14.043948 16236 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "300d9788081b429faac41e293d8f37dc" candidate_uuid: "fc6dfd46f9cc4201886c620db1510312" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85786fb38f3f4634ae1b440c35d07d3a"
I20260810 11:31:14.044185 16236 raft_consensus.cc:3037] T 300d9788081b429faac41e293d8f37dc P 85786fb38f3f4634ae1b440c35d07d3a [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:31:14.044214 16384 raft_consensus.cc:3037] T 300d9788081b429faac41e293d8f37dc P 83fb5fdfb5e34ed399aad0e87934d93c [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:31:14.047377 16236 raft_consensus.cc:2454] T 300d9788081b429faac41e293d8f37dc P 85786fb38f3f4634ae1b440c35d07d3a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc6dfd46f9cc4201886c620db1510312 in term 1.
I20260810 11:31:14.047744 16384 raft_consensus.cc:2454] T 300d9788081b429faac41e293d8f37dc P 83fb5fdfb5e34ed399aad0e87934d93c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc6dfd46f9cc4201886c620db1510312 in term 1.
I20260810 11:31:14.048942 16459 leader_election.cc:302] T 300d9788081b429faac41e293d8f37dc P fc6dfd46f9cc4201886c620db1510312 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83fb5fdfb5e34ed399aad0e87934d93c, fc6dfd46f9cc4201886c620db1510312; no voters:
I20260810 11:31:14.049856 16616 raft_consensus.cc:2781] T 300d9788081b429faac41e293d8f37dc P fc6dfd46f9cc4201886c620db1510312 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:31:14.050452 16616 raft_consensus.cc:686] T 300d9788081b429faac41e293d8f37dc P fc6dfd46f9cc4201886c620db1510312 [term 1 LEADER]: Becoming Leader. State: Replica: fc6dfd46f9cc4201886c620db1510312, State: Running, Role: LEADER
I20260810 11:31:14.050916 16616 consensus_queue.cc:237] T 300d9788081b429faac41e293d8f37dc P fc6dfd46f9cc4201886c620db1510312 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85786fb38f3f4634ae1b440c35d07d3a" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 41221 } } peers { permanent_uuid: "83fb5fdfb5e34ed399aad0e87934d93c" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 45817 } } peers { permanent_uuid: "fc6dfd46f9cc4201886c620db1510312" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 35295 } }
I20260810 11:31:14.055183 16074 catalog_manager.cc:5495] T 300d9788081b429faac41e293d8f37dc P fc6dfd46f9cc4201886c620db1510312 reported cstate change: term changed from 0 to 1, leader changed from <none> to fc6dfd46f9cc4201886c620db1510312 (127.8.18.195). New cstate: current_term: 1 leader_uuid: "fc6dfd46f9cc4201886c620db1510312" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85786fb38f3f4634ae1b440c35d07d3a" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 41221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "83fb5fdfb5e34ed399aad0e87934d93c" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 45817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc6dfd46f9cc4201886c620db1510312" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 35295 } health_report { overall_health: HEALTHY } } }
I20260810 11:31:14.084652 16682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:14.077162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56738 (local address 127.8.18.195:35295)
0810 11:31:14.077877 (+ 715us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:14.077882 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:14.077903 (+ 21us) server_negotiation.cc:407] Connection header received
0810 11:31:14.077998 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:14.078002 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:14.078081 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:14.078250 (+ 169us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 11:31:14.078738 (+ 488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:14.079587 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:14.081787 (+ 2200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:14.082306 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:14.082425 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:14.083068 (+ 643us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 11:31:14.083753 (+ 685us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 11:31:14.083882 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0810 11:31:14.084053 (+ 171us) server_negotiation.cc:299] Negotiation successful
0810 11:31:14.084140 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":600,"thread_start_us":439,"threads_started":1}
I20260810 11:31:14.089002 16236 raft_consensus.cc:1261] T 300d9788081b429faac41e293d8f37dc P 85786fb38f3f4634ae1b440c35d07d3a [term 1 FOLLOWER]: Refusing update from remote peer fc6dfd46f9cc4201886c620db1510312: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 11:31:14.089316 16384 raft_consensus.cc:1261] T 300d9788081b429faac41e293d8f37dc P 83fb5fdfb5e34ed399aad0e87934d93c [term 1 FOLLOWER]: Refusing update from remote peer fc6dfd46f9cc4201886c620db1510312: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 11:31:14.089828 16616 consensus_queue.cc:1035] T 300d9788081b429faac41e293d8f37dc P fc6dfd46f9cc4201886c620db1510312 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85786fb38f3f4634ae1b440c35d07d3a" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 41221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 11:31:14.090214 16679 consensus_queue.cc:1035] T 300d9788081b429faac41e293d8f37dc P fc6dfd46f9cc4201886c620db1510312 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83fb5fdfb5e34ed399aad0e87934d93c" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 45817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 11:31:14.116107 16075 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50958:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 11:31:14.116451 16075 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50958:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 11:31:14.120471 16075 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 064ed84ccdb040029c1070bb3cbfe22c: Sending DeleteTablet for 3 replicas of tablet 300d9788081b429faac41e293d8f37dc
I20260810 11:31:14.122013 16215 tablet_service.cc:1514] Processing DeleteTablet for tablet 300d9788081b429faac41e293d8f37dc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:31:14 UTC) from {username='oryx', principal='oryx/127.8.18.254@KRBTEST.COM'} at 127.0.0.1:36068
I20260810 11:31:14.122447 16509 tablet_service.cc:1514] Processing DeleteTablet for tablet 300d9788081b429faac41e293d8f37dc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:31:14 UTC) from {username='oryx', principal='oryx/127.8.18.254@KRBTEST.COM'} at 127.0.0.1:56726
I20260810 11:31:14.122888 16634 tablet_replica.cc:330] T 300d9788081b429faac41e293d8f37dc P 85786fb38f3f4634ae1b440c35d07d3a: stopping tablet replica
I20260810 11:31:14.123005 16362 tablet_service.cc:1514] Processing DeleteTablet for tablet 300d9788081b429faac41e293d8f37dc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:31:14 UTC) from {username='oryx', principal='oryx/127.8.18.254@KRBTEST.COM'} at 127.0.0.1:51318
I20260810 11:31:14.123096 16634 raft_consensus.cc:2229] T 300d9788081b429faac41e293d8f37dc P 85786fb38f3f4634ae1b440c35d07d3a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 11:31:14.123399 16632 tablet_replica.cc:330] T 300d9788081b429faac41e293d8f37dc P fc6dfd46f9cc4201886c620db1510312: stopping tablet replica
I20260810 11:31:14.123396 16634 raft_consensus.cc:2258] T 300d9788081b429faac41e293d8f37dc P 85786fb38f3f4634ae1b440c35d07d3a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:31:14.123602 16632 raft_consensus.cc:2229] T 300d9788081b429faac41e293d8f37dc P fc6dfd46f9cc4201886c620db1510312 [term 1 LEADER]: Raft consensus shutting down.
I20260810 11:31:14.124250 16632 raft_consensus.cc:2258] T 300d9788081b429faac41e293d8f37dc P fc6dfd46f9cc4201886c620db1510312 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:31:14.124627 16633 tablet_replica.cc:330] T 300d9788081b429faac41e293d8f37dc P 83fb5fdfb5e34ed399aad0e87934d93c: stopping tablet replica
I20260810 11:31:14.124763 16634 ts_tablet_manager.cc:1905] T 300d9788081b429faac41e293d8f37dc P 85786fb38f3f4634ae1b440c35d07d3a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:31:14.124830 16633 raft_consensus.cc:2229] T 300d9788081b429faac41e293d8f37dc P 83fb5fdfb5e34ed399aad0e87934d93c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 11:31:14.125093 16633 raft_consensus.cc:2258] T 300d9788081b429faac41e293d8f37dc P 83fb5fdfb5e34ed399aad0e87934d93c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:31:14.127803 16633 ts_tablet_manager.cc:1905] T 300d9788081b429faac41e293d8f37dc P 83fb5fdfb5e34ed399aad0e87934d93c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:31:14.130546 16632 ts_tablet_manager.cc:1905] T 300d9788081b429faac41e293d8f37dc P fc6dfd46f9cc4201886c620db1510312: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:31:14.133455 16634 ts_tablet_manager.cc:1918] T 300d9788081b429faac41e293d8f37dc P 85786fb38f3f4634ae1b440c35d07d3a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 11:31:14.133692 16634 log.cc:1196] T 300d9788081b429faac41e293d8f37dc P 85786fb38f3f4634ae1b440c35d07d3a: Deleting WAL directory at /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786361401922362-8267-0/minicluster-data/ts-0/wal/wals/300d9788081b429faac41e293d8f37dc
I20260810 11:31:14.134188 16634 ts_tablet_manager.cc:1939] T 300d9788081b429faac41e293d8f37dc P 85786fb38f3f4634ae1b440c35d07d3a: Deleting consensus metadata
I20260810 11:31:14.135682 16060 catalog_manager.cc:4841] TS 85786fb38f3f4634ae1b440c35d07d3a (127.8.18.193:41221): tablet 300d9788081b429faac41e293d8f37dc (table test-table [id=e54a35124c094b9484438e06bcffb374]) successfully deleted
I20260810 11:31:14.135658 16075 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50958:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 11:31:14.140503 16633 ts_tablet_manager.cc:1918] T 300d9788081b429faac41e293d8f37dc P 83fb5fdfb5e34ed399aad0e87934d93c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 11:31:14.140686 16633 log.cc:1196] T 300d9788081b429faac41e293d8f37dc P 83fb5fdfb5e34ed399aad0e87934d93c: Deleting WAL directory at /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786361401922362-8267-0/minicluster-data/ts-1/wal/wals/300d9788081b429faac41e293d8f37dc
I20260810 11:31:14.141148 16633 ts_tablet_manager.cc:1939] T 300d9788081b429faac41e293d8f37dc P 83fb5fdfb5e34ed399aad0e87934d93c: Deleting consensus metadata
W20260810 11:31:14.141428 16075 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 11:31:14.142552 16060 catalog_manager.cc:4841] TS 83fb5fdfb5e34ed399aad0e87934d93c (127.8.18.194:45817): tablet 300d9788081b429faac41e293d8f37dc (table test-table [id=e54a35124c094b9484438e06bcffb374]) successfully deleted
I20260810 11:31:14.145398 16632 ts_tablet_manager.cc:1918] T 300d9788081b429faac41e293d8f37dc P fc6dfd46f9cc4201886c620db1510312: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 11:31:14.145673 16632 log.cc:1196] T 300d9788081b429faac41e293d8f37dc P fc6dfd46f9cc4201886c620db1510312: Deleting WAL directory at /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786361401922362-8267-0/minicluster-data/ts-2/wal/wals/300d9788081b429faac41e293d8f37dc
I20260810 11:31:14.146174 16632 ts_tablet_manager.cc:1939] T 300d9788081b429faac41e293d8f37dc P fc6dfd46f9cc4201886c620db1510312: Deleting consensus metadata
I20260810 11:31:14.147624 16060 catalog_manager.cc:4841] TS fc6dfd46f9cc4201886c620db1510312 (127.8.18.195:35295): tablet 300d9788081b429faac41e293d8f37dc (table test-table [id=e54a35124c094b9484438e06bcffb374]) successfully deleted
I20260810 11:31:14.151661 16215 tablet_service.cc:1467] Processing CreateTablet for tablet 3b448f25ae554eda8e86cbc4fefdc67a (DEFAULT_TABLE table=test-table [id=dea74d5189da4afeb7fa790490a3b5b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:31:14.151700 16509 tablet_service.cc:1467] Processing CreateTablet for tablet 3b448f25ae554eda8e86cbc4fefdc67a (DEFAULT_TABLE table=test-table [id=dea74d5189da4afeb7fa790490a3b5b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:31:14.152170 16509 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3b448f25ae554eda8e86cbc4fefdc67a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:31:14.152189 16215 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3b448f25ae554eda8e86cbc4fefdc67a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:31:14.153883 16362 tablet_service.cc:1467] Processing CreateTablet for tablet 3b448f25ae554eda8e86cbc4fefdc67a (DEFAULT_TABLE table=test-table [id=dea74d5189da4afeb7fa790490a3b5b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:31:14.154346 16362 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3b448f25ae554eda8e86cbc4fefdc67a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:31:14.159904 16609 tablet_bootstrap.cc:492] T 3b448f25ae554eda8e86cbc4fefdc67a P fc6dfd46f9cc4201886c620db1510312: Bootstrap starting.
I20260810 11:31:14.161970 16609 tablet_bootstrap.cc:654] T 3b448f25ae554eda8e86cbc4fefdc67a P fc6dfd46f9cc4201886c620db1510312: Neither blocks nor log segments found. Creating new log.
I20260810 11:31:14.172075 16608 tablet_bootstrap.cc:492] T 3b448f25ae554eda8e86cbc4fefdc67a P 83fb5fdfb5e34ed399aad0e87934d93c: Bootstrap starting.
I20260810 11:31:14.173161 16609 tablet_bootstrap.cc:492] T 3b448f25ae554eda8e86cbc4fefdc67a P fc6dfd46f9cc4201886c620db1510312: No bootstrap required, opened a new log
I20260810 11:31:14.173355 16609 ts_tablet_manager.cc:1397] T 3b448f25ae554eda8e86cbc4fefdc67a P fc6dfd46f9cc4201886c620db1510312: Time spent bootstrapping tablet: real 0.014s user 0.006s sys 0.004s
I20260810 11:31:14.174247 16608 tablet_bootstrap.cc:654] T 3b448f25ae554eda8e86cbc4fefdc67a P 83fb5fdfb5e34ed399aad0e87934d93c: Neither blocks nor log segments found. Creating new log.
I20260810 11:31:14.174311 16609 raft_consensus.cc:348] T 3b448f25ae554eda8e86cbc4fefdc67a P fc6dfd46f9cc4201886c620db1510312 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85786fb38f3f4634ae1b440c35d07d3a" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 41221 } } peers { permanent_uuid: "83fb5fdfb5e34ed399aad0e87934d93c" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 45817 } } peers { permanent_uuid: "fc6dfd46f9cc4201886c620db1510312" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 35295 } }
I20260810 11:31:14.174606 16609 raft_consensus.cc:374] T 3b448f25ae554eda8e86cbc4fefdc67a P fc6dfd46f9cc4201886c620db1510312 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:31:14.174718 16609 raft_consensus.cc:729] T 3b448f25ae554eda8e86cbc4fefdc67a P fc6dfd46f9cc4201886c620db1510312 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc6dfd46f9cc4201886c620db1510312, State: Initialized, Role: FOLLOWER
I20260810 11:31:14.175139 16607 tablet_bootstrap.cc:492] T 3b448f25ae554eda8e86cbc4fefdc67a P 85786fb38f3f4634ae1b440c35d07d3a: Bootstrap starting.
I20260810 11:31:14.174998 16609 consensus_queue.cc:260] T 3b448f25ae554eda8e86cbc4fefdc67a P fc6dfd46f9cc4201886c620db1510312 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85786fb38f3f4634ae1b440c35d07d3a" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 41221 } } peers { permanent_uuid: "83fb5fdfb5e34ed399aad0e87934d93c" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 45817 } } peers { permanent_uuid: "fc6dfd46f9cc4201886c620db1510312" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 35295 } }
I20260810 11:31:14.175539 16609 ts_tablet_manager.cc:1428] T 3b448f25ae554eda8e86cbc4fefdc67a P fc6dfd46f9cc4201886c620db1510312: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260810 11:31:14.176962 16607 tablet_bootstrap.cc:654] T 3b448f25ae554eda8e86cbc4fefdc67a P 85786fb38f3f4634ae1b440c35d07d3a: Neither blocks nor log segments found. Creating new log.
I20260810 11:31:14.185110 16607 tablet_bootstrap.cc:492] T 3b448f25ae554eda8e86cbc4fefdc67a P 85786fb38f3f4634ae1b440c35d07d3a: No bootstrap required, opened a new log
I20260810 11:31:14.185125 16608 tablet_bootstrap.cc:492] T 3b448f25ae554eda8e86cbc4fefdc67a P 83fb5fdfb5e34ed399aad0e87934d93c: No bootstrap required, opened a new log
I20260810 11:31:14.185367 16607 ts_tablet_manager.cc:1397] T 3b448f25ae554eda8e86cbc4fefdc67a P 85786fb38f3f4634ae1b440c35d07d3a: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.003s
I20260810 11:31:14.185385 16608 ts_tablet_manager.cc:1397] T 3b448f25ae554eda8e86cbc4fefdc67a P 83fb5fdfb5e34ed399aad0e87934d93c: Time spent bootstrapping tablet: real 0.013s user 0.003s sys 0.003s
I20260810 11:31:14.186349 16608 raft_consensus.cc:348] T 3b448f25ae554eda8e86cbc4fefdc67a P 83fb5fdfb5e34ed399aad0e87934d93c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85786fb38f3f4634ae1b440c35d07d3a" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 41221 } } peers { permanent_uuid: "83fb5fdfb5e34ed399aad0e87934d93c" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 45817 } } peers { permanent_uuid: "fc6dfd46f9cc4201886c620db1510312" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 35295 } }
I20260810 11:31:14.186663 16608 raft_consensus.cc:374] T 3b448f25ae554eda8e86cbc4fefdc67a P 83fb5fdfb5e34ed399aad0e87934d93c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:31:14.186772 16608 raft_consensus.cc:729] T 3b448f25ae554eda8e86cbc4fefdc67a P 83fb5fdfb5e34ed399aad0e87934d93c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83fb5fdfb5e34ed399aad0e87934d93c, State: Initialized, Role: FOLLOWER
I20260810 11:31:14.187096 16608 consensus_queue.cc:260] T 3b448f25ae554eda8e86cbc4fefdc67a P 83fb5fdfb5e34ed399aad0e87934d93c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85786fb38f3f4634ae1b440c35d07d3a" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 41221 } } peers { permanent_uuid: "83fb5fdfb5e34ed399aad0e87934d93c" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 45817 } } peers { permanent_uuid: "fc6dfd46f9cc4201886c620db1510312" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 35295 } }
I20260810 11:31:14.188019 16607 raft_consensus.cc:348] T 3b448f25ae554eda8e86cbc4fefdc67a P 85786fb38f3f4634ae1b440c35d07d3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85786fb38f3f4634ae1b440c35d07d3a" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 41221 } } peers { permanent_uuid: "83fb5fdfb5e34ed399aad0e87934d93c" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 45817 } } peers { permanent_uuid: "fc6dfd46f9cc4201886c620db1510312" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 35295 } }
I20260810 11:31:14.188315 16607 raft_consensus.cc:374] T 3b448f25ae554eda8e86cbc4fefdc67a P 85786fb38f3f4634ae1b440c35d07d3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:31:14.188413 16607 raft_consensus.cc:729] T 3b448f25ae554eda8e86cbc4fefdc67a P 85786fb38f3f4634ae1b440c35d07d3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85786fb38f3f4634ae1b440c35d07d3a, State: Initialized, Role: FOLLOWER
I20260810 11:31:14.188721 16607 consensus_queue.cc:260] T 3b448f25ae554eda8e86cbc4fefdc67a P 85786fb38f3f4634ae1b440c35d07d3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85786fb38f3f4634ae1b440c35d07d3a" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 41221 } } peers { permanent_uuid: "83fb5fdfb5e34ed399aad0e87934d93c" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 45817 } } peers { permanent_uuid: "fc6dfd46f9cc4201886c620db1510312" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 35295 } }
I20260810 11:31:14.190079 16607 ts_tablet_manager.cc:1428] T 3b448f25ae554eda8e86cbc4fefdc67a P 85786fb38f3f4634ae1b440c35d07d3a: Time spent starting tablet: real 0.005s user 0.000s sys 0.002s
I20260810 11:31:14.190622 16608 ts_tablet_manager.cc:1428] T 3b448f25ae554eda8e86cbc4fefdc67a P 83fb5fdfb5e34ed399aad0e87934d93c: Time spent starting tablet: real 0.005s user 0.000s sys 0.002s
I20260810 11:31:14.474120 16616 raft_consensus.cc:482] T 3b448f25ae554eda8e86cbc4fefdc67a P fc6dfd46f9cc4201886c620db1510312 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 11:31:14.474382 16616 raft_consensus.cc:504] T 3b448f25ae554eda8e86cbc4fefdc67a P fc6dfd46f9cc4201886c620db1510312 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85786fb38f3f4634ae1b440c35d07d3a" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 41221 } } peers { permanent_uuid: "83fb5fdfb5e34ed399aad0e87934d93c" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 45817 } } peers { permanent_uuid: "fc6dfd46f9cc4201886c620db1510312" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 35295 } }
I20260810 11:31:14.475067 16616 leader_election.cc:288] T 3b448f25ae554eda8e86cbc4fefdc67a P fc6dfd46f9cc4201886c620db1510312 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 85786fb38f3f4634ae1b440c35d07d3a (127.8.18.193:41221), 83fb5fdfb5e34ed399aad0e87934d93c (127.8.18.194:45817)
I20260810 11:31:14.475766 16236 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b448f25ae554eda8e86cbc4fefdc67a" candidate_uuid: "fc6dfd46f9cc4201886c620db1510312" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85786fb38f3f4634ae1b440c35d07d3a" is_pre_election: true
I20260810 11:31:14.475813 16383 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b448f25ae554eda8e86cbc4fefdc67a" candidate_uuid: "fc6dfd46f9cc4201886c620db1510312" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83fb5fdfb5e34ed399aad0e87934d93c" is_pre_election: true
I20260810 11:31:14.476050 16236 raft_consensus.cc:2454] T 3b448f25ae554eda8e86cbc4fefdc67a P 85786fb38f3f4634ae1b440c35d07d3a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc6dfd46f9cc4201886c620db1510312 in term 0.
I20260810 11:31:14.476153 16383 raft_consensus.cc:2454] T 3b448f25ae554eda8e86cbc4fefdc67a P 83fb5fdfb5e34ed399aad0e87934d93c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc6dfd46f9cc4201886c620db1510312 in term 0.
I20260810 11:31:14.476615 16459 leader_election.cc:302] T 3b448f25ae554eda8e86cbc4fefdc67a P fc6dfd46f9cc4201886c620db1510312 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85786fb38f3f4634ae1b440c35d07d3a, fc6dfd46f9cc4201886c620db1510312; no voters:
I20260810 11:31:14.476933 16616 raft_consensus.cc:2781] T 3b448f25ae554eda8e86cbc4fefdc67a P fc6dfd46f9cc4201886c620db1510312 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 11:31:14.477075 16616 raft_consensus.cc:482] T 3b448f25ae554eda8e86cbc4fefdc67a P fc6dfd46f9cc4201886c620db1510312 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 11:31:14.477201 16616 raft_consensus.cc:3037] T 3b448f25ae554eda8e86cbc4fefdc67a P fc6dfd46f9cc4201886c620db1510312 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:31:14.479660 16616 raft_consensus.cc:504] T 3b448f25ae554eda8e86cbc4fefdc67a P fc6dfd46f9cc4201886c620db1510312 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85786fb38f3f4634ae1b440c35d07d3a" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 41221 } } peers { permanent_uuid: "83fb5fdfb5e34ed399aad0e87934d93c" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 45817 } } peers { permanent_uuid: "fc6dfd46f9cc4201886c620db1510312" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 35295 } }
I20260810 11:31:14.480326 16616 leader_election.cc:288] T 3b448f25ae554eda8e86cbc4fefdc67a P fc6dfd46f9cc4201886c620db1510312 [CANDIDATE]: Term 1 election: Requested vote from peers 85786fb38f3f4634ae1b440c35d07d3a (127.8.18.193:41221), 83fb5fdfb5e34ed399aad0e87934d93c (127.8.18.194:45817)
I20260810 11:31:14.481074 16383 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b448f25ae554eda8e86cbc4fefdc67a" candidate_uuid: "fc6dfd46f9cc4201886c620db1510312" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83fb5fdfb5e34ed399aad0e87934d93c"
I20260810 11:31:14.481120 16236 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b448f25ae554eda8e86cbc4fefdc67a" candidate_uuid: "fc6dfd46f9cc4201886c620db1510312" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85786fb38f3f4634ae1b440c35d07d3a"
I20260810 11:31:14.481331 16383 raft_consensus.cc:3037] T 3b448f25ae554eda8e86cbc4fefdc67a P 83fb5fdfb5e34ed399aad0e87934d93c [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:31:14.481385 16236 raft_consensus.cc:3037] T 3b448f25ae554eda8e86cbc4fefdc67a P 85786fb38f3f4634ae1b440c35d07d3a [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:31:14.484670 16236 raft_consensus.cc:2454] T 3b448f25ae554eda8e86cbc4fefdc67a P 85786fb38f3f4634ae1b440c35d07d3a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc6dfd46f9cc4201886c620db1510312 in term 1.
I20260810 11:31:14.484670 16383 raft_consensus.cc:2454] T 3b448f25ae554eda8e86cbc4fefdc67a P 83fb5fdfb5e34ed399aad0e87934d93c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc6dfd46f9cc4201886c620db1510312 in term 1.
I20260810 11:31:14.485314 16459 leader_election.cc:302] T 3b448f25ae554eda8e86cbc4fefdc67a P fc6dfd46f9cc4201886c620db1510312 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85786fb38f3f4634ae1b440c35d07d3a, fc6dfd46f9cc4201886c620db1510312; no voters:
I20260810 11:31:14.485720 16616 raft_consensus.cc:2781] T 3b448f25ae554eda8e86cbc4fefdc67a P fc6dfd46f9cc4201886c620db1510312 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:31:14.485932 16616 raft_consensus.cc:686] T 3b448f25ae554eda8e86cbc4fefdc67a P fc6dfd46f9cc4201886c620db1510312 [term 1 LEADER]: Becoming Leader. State: Replica: fc6dfd46f9cc4201886c620db1510312, State: Running, Role: LEADER
I20260810 11:31:14.486231 16616 consensus_queue.cc:237] T 3b448f25ae554eda8e86cbc4fefdc67a P fc6dfd46f9cc4201886c620db1510312 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85786fb38f3f4634ae1b440c35d07d3a" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 41221 } } peers { permanent_uuid: "83fb5fdfb5e34ed399aad0e87934d93c" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 45817 } } peers { permanent_uuid: "fc6dfd46f9cc4201886c620db1510312" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 35295 } }
I20260810 11:31:14.489506 16073 catalog_manager.cc:5495] T 3b448f25ae554eda8e86cbc4fefdc67a P fc6dfd46f9cc4201886c620db1510312 reported cstate change: term changed from 0 to 1, leader changed from <none> to fc6dfd46f9cc4201886c620db1510312 (127.8.18.195). New cstate: current_term: 1 leader_uuid: "fc6dfd46f9cc4201886c620db1510312" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85786fb38f3f4634ae1b440c35d07d3a" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 41221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "83fb5fdfb5e34ed399aad0e87934d93c" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 45817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc6dfd46f9cc4201886c620db1510312" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 35295 } health_report { overall_health: HEALTHY } } }
I20260810 11:31:14.579913 16695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:14.572688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.194:45817 (local address 127.8.18.195:60071)
0810 11:31:14.573444 (+ 756us) negotiation.cc:107] Waiting for socket to connect
0810 11:31:14.573471 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0810 11:31:14.573657 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:31:14.574157 (+ 500us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:31:14.574160 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:31:14.574170 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 11:31:14.574426 (+ 256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:14.574435 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:14.576067 (+ 1632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:14.576072 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:14.577342 (+ 1270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:14.577364 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:14.578639 (+ 1275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:14.578644 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:14.578783 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:14.578852 (+ 69us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 11:31:14.579367 (+ 515us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 11:31:14.579378 (+ 11us) client_negotiation.cc:769] Sending connection context
0810 11:31:14.579453 (+ 75us) client_negotiation.cc:240] Negotiation successful
0810 11:31:14.579580 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":519,"thread_start_us":591,"threads_started":1}
I20260810 11:31:14.579984 16696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:14.573132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.195:60071 (local address 127.8.18.194:45817)
0810 11:31:14.573774 (+ 642us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:14.573781 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:14.573808 (+ 27us) server_negotiation.cc:407] Connection header received
0810 11:31:14.573894 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:14.573900 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:14.574002 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:14.574137 (+ 135us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 11:31:14.574637 (+ 500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:14.575874 (+ 1237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:14.577578 (+ 1704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:14.578402 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:14.578562 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:14.579069 (+ 507us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 11:31:14.579251 (+ 182us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 11:31:14.579354 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0810 11:31:14.579508 (+ 154us) server_negotiation.cc:299] Negotiation successful
0810 11:31:14.579621 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":499,"thread_start_us":394,"threads_started":1}
I20260810 11:31:14.591034 16383 raft_consensus.cc:1261] T 3b448f25ae554eda8e86cbc4fefdc67a P 83fb5fdfb5e34ed399aad0e87934d93c [term 1 FOLLOWER]: Refusing update from remote peer fc6dfd46f9cc4201886c620db1510312: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 11:31:14.591099 16236 raft_consensus.cc:1261] T 3b448f25ae554eda8e86cbc4fefdc67a P 85786fb38f3f4634ae1b440c35d07d3a [term 1 FOLLOWER]: Refusing update from remote peer fc6dfd46f9cc4201886c620db1510312: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 11:31:14.591701 16616 consensus_queue.cc:1035] T 3b448f25ae554eda8e86cbc4fefdc67a P fc6dfd46f9cc4201886c620db1510312 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85786fb38f3f4634ae1b440c35d07d3a" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 41221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 11:31:14.592245 16679 consensus_queue.cc:1035] T 3b448f25ae554eda8e86cbc4fefdc67a P fc6dfd46f9cc4201886c620db1510312 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83fb5fdfb5e34ed399aad0e87934d93c" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 45817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 11:31:14.632596 16701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:14.621518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.195:35295 (local address 127.8.18.194:56361)
0810 11:31:14.622128 (+ 610us) negotiation.cc:107] Waiting for socket to connect
0810 11:31:14.622160 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0810 11:31:14.622326 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:31:14.626714 (+ 4388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:31:14.626721 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:31:14.626740 (+ 19us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 11:31:14.627088 (+ 348us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:14.627101 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:14.628841 (+ 1740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:14.628847 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:14.630265 (+ 1418us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:14.630281 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:14.631445 (+ 1164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:14.631450 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:14.631587 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:14.631629 (+ 42us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 11:31:14.632058 (+ 429us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 11:31:14.632067 (+ 9us) client_negotiation.cc:769] Sending connection context
0810 11:31:14.632126 (+ 59us) client_negotiation.cc:240] Negotiation successful
0810 11:31:14.632287 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":440,"thread_start_us":468,"threads_started":1}
I20260810 11:31:14.632725 16702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:14.621823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.194:56361 (local address 127.8.18.195:35295)
0810 11:31:14.626374 (+ 4551us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:14.626380 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:14.626406 (+ 26us) server_negotiation.cc:407] Connection header received
0810 11:31:14.626476 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:14.626481 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:14.626565 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:14.626687 (+ 122us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 11:31:14.627278 (+ 591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:14.628645 (+ 1367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:14.630429 (+ 1784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:14.631168 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:14.631329 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:14.631808 (+ 479us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 11:31:14.631965 (+ 157us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 11:31:14.632044 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0810 11:31:14.632163 (+ 119us) server_negotiation.cc:299] Negotiation successful
0810 11:31:14.632312 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4407,"thread_start_us":4200,"threads_started":1}
I20260810 11:31:14.668651 16074 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50958:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 11:31:14.668973 16074 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50958:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 11:31:14.671967 16074 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 064ed84ccdb040029c1070bb3cbfe22c: Sending DeleteTablet for 3 replicas of tablet 3b448f25ae554eda8e86cbc4fefdc67a
I20260810 11:31:14.672878 16215 tablet_service.cc:1514] Processing DeleteTablet for tablet 3b448f25ae554eda8e86cbc4fefdc67a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:31:14 UTC) from {username='oryx', principal='oryx/127.8.18.254@KRBTEST.COM'} at 127.0.0.1:36068
I20260810 11:31:14.672920 16362 tablet_service.cc:1514] Processing DeleteTablet for tablet 3b448f25ae554eda8e86cbc4fefdc67a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:31:14 UTC) from {username='oryx', principal='oryx/127.8.18.254@KRBTEST.COM'} at 127.0.0.1:51318
I20260810 11:31:14.672945 16509 tablet_service.cc:1514] Processing DeleteTablet for tablet 3b448f25ae554eda8e86cbc4fefdc67a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:31:14 UTC) from {username='oryx', principal='oryx/127.8.18.254@KRBTEST.COM'} at 127.0.0.1:56726
I20260810 11:31:14.673727 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 16148
I20260810 11:31:14.689160 16705 tablet_replica.cc:330] T 3b448f25ae554eda8e86cbc4fefdc67a P fc6dfd46f9cc4201886c620db1510312: stopping tablet replica
I20260810 11:31:14.689509 16705 raft_consensus.cc:2229] T 3b448f25ae554eda8e86cbc4fefdc67a P fc6dfd46f9cc4201886c620db1510312 [term 1 LEADER]: Raft consensus shutting down.
I20260810 11:31:14.689870 16704 tablet_replica.cc:330] T 3b448f25ae554eda8e86cbc4fefdc67a P 85786fb38f3f4634ae1b440c35d07d3a: stopping tablet replica
I20260810 11:31:14.689973 16705 raft_consensus.cc:2258] T 3b448f25ae554eda8e86cbc4fefdc67a P fc6dfd46f9cc4201886c620db1510312 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:31:14.690099 16704 raft_consensus.cc:2229] T 3b448f25ae554eda8e86cbc4fefdc67a P 85786fb38f3f4634ae1b440c35d07d3a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 11:31:14.690727 16704 raft_consensus.cc:2258] T 3b448f25ae554eda8e86cbc4fefdc67a P 85786fb38f3f4634ae1b440c35d07d3a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:31:14.691778 16705 ts_tablet_manager.cc:1905] T 3b448f25ae554eda8e86cbc4fefdc67a P fc6dfd46f9cc4201886c620db1510312: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:31:14.692103 16704 ts_tablet_manager.cc:1905] T 3b448f25ae554eda8e86cbc4fefdc67a P 85786fb38f3f4634ae1b440c35d07d3a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:31:14.693845 16706 tablet_replica.cc:330] T 3b448f25ae554eda8e86cbc4fefdc67a P 83fb5fdfb5e34ed399aad0e87934d93c: stopping tablet replica
I20260810 11:31:14.694099 16706 raft_consensus.cc:2229] T 3b448f25ae554eda8e86cbc4fefdc67a P 83fb5fdfb5e34ed399aad0e87934d93c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 11:31:14.694361 16706 raft_consensus.cc:2258] T 3b448f25ae554eda8e86cbc4fefdc67a P 83fb5fdfb5e34ed399aad0e87934d93c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:31:14.695544 16706 ts_tablet_manager.cc:1905] T 3b448f25ae554eda8e86cbc4fefdc67a P 83fb5fdfb5e34ed399aad0e87934d93c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:31:14.699532 16704 ts_tablet_manager.cc:1918] T 3b448f25ae554eda8e86cbc4fefdc67a P 85786fb38f3f4634ae1b440c35d07d3a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 11:31:14.699720 16704 log.cc:1196] T 3b448f25ae554eda8e86cbc4fefdc67a P 85786fb38f3f4634ae1b440c35d07d3a: Deleting WAL directory at /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786361401922362-8267-0/minicluster-data/ts-0/wal/wals/3b448f25ae554eda8e86cbc4fefdc67a
I20260810 11:31:14.700245 16704 ts_tablet_manager.cc:1939] T 3b448f25ae554eda8e86cbc4fefdc67a P 85786fb38f3f4634ae1b440c35d07d3a: Deleting consensus metadata
I20260810 11:31:14.700275 16705 ts_tablet_manager.cc:1918] T 3b448f25ae554eda8e86cbc4fefdc67a P fc6dfd46f9cc4201886c620db1510312: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 11:31:14.700593 16705 log.cc:1196] T 3b448f25ae554eda8e86cbc4fefdc67a P fc6dfd46f9cc4201886c620db1510312: Deleting WAL directory at /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786361401922362-8267-0/minicluster-data/ts-2/wal/wals/3b448f25ae554eda8e86cbc4fefdc67a
I20260810 11:31:14.701208 16705 ts_tablet_manager.cc:1939] T 3b448f25ae554eda8e86cbc4fefdc67a P fc6dfd46f9cc4201886c620db1510312: Deleting consensus metadata
I20260810 11:31:14.702714 16060 catalog_manager.cc:4841] TS fc6dfd46f9cc4201886c620db1510312 (127.8.18.195:35295): tablet 3b448f25ae554eda8e86cbc4fefdc67a (table test-table [id=dea74d5189da4afeb7fa790490a3b5b7]) successfully deleted
I20260810 11:31:14.703086 16060 catalog_manager.cc:4841] TS 85786fb38f3f4634ae1b440c35d07d3a (127.8.18.193:41221): tablet 3b448f25ae554eda8e86cbc4fefdc67a (table test-table [id=dea74d5189da4afeb7fa790490a3b5b7]) successfully deleted
I20260810 11:31:14.703577 16706 ts_tablet_manager.cc:1918] T 3b448f25ae554eda8e86cbc4fefdc67a P 83fb5fdfb5e34ed399aad0e87934d93c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 11:31:14.703778 16706 log.cc:1196] T 3b448f25ae554eda8e86cbc4fefdc67a P 83fb5fdfb5e34ed399aad0e87934d93c: Deleting WAL directory at /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786361401922362-8267-0/minicluster-data/ts-1/wal/wals/3b448f25ae554eda8e86cbc4fefdc67a
I20260810 11:31:14.704221 16706 ts_tablet_manager.cc:1939] T 3b448f25ae554eda8e86cbc4fefdc67a P 83fb5fdfb5e34ed399aad0e87934d93c: Deleting consensus metadata
I20260810 11:31:14.705513 16060 catalog_manager.cc:4841] TS 83fb5fdfb5e34ed399aad0e87934d93c (127.8.18.194:45817): tablet 3b448f25ae554eda8e86cbc4fefdc67a (table test-table [id=dea74d5189da4afeb7fa790490a3b5b7]) successfully deleted
Aug 10 11:31:14 dist-test-slave-srgv krb5kdc[16029](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361473, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.8.18.193@KRBTEST.COM
I20260810 11:31:14.713327 16708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:14.674978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36076 (local address 127.8.18.193:41221)
0810 11:31:14.694149 (+ 19171us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:14.694155 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:14.694191 (+ 36us) server_negotiation.cc:407] Connection header received
0810 11:31:14.694262 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:14.694269 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:14.694368 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:14.694499 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:31:14.696828 (+ 2329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:14.698111 (+ 1283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:14.700127 (+ 2016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:14.700897 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:14.701074 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:14.706965 (+ 5891us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:31:14.706983 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:31:14.707001 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:31:14.707049 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:31:14.710366 (+ 3317us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:31:14.711299 (+ 933us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:31:14.711309 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:31:14.711312 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:31:14.711464 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:31:14.711862 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:31:14.711865 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:31:14.711867 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:31:14.712299 (+ 432us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:31:14.712523 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0810 11:31:14.712834 (+ 311us) server_negotiation.cc:299] Negotiation successful
0810 11:31:14.713038 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19008,"spinlock_wait_cycles":2944,"thread_start_us":18962,"threads_started":1}
I20260810 11:31:14.714421 16284 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:31:14.815846 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 16148
I20260810 11:31:14.834004 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 16294
Aug 10 11:31:14 dist-test-slave-srgv krb5kdc[16029](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361473, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.8.18.194@KRBTEST.COM
I20260810 11:31:14.851459 16696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:14.834864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51344 (local address 127.8.18.194:45817)
0810 11:31:14.835046 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:14.835051 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:14.835074 (+ 23us) server_negotiation.cc:407] Connection header received
0810 11:31:14.835305 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:14.835309 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:14.835384 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:14.835515 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:31:14.836819 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:14.837723 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:14.839070 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:14.839545 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:14.839661 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:14.844474 (+ 4813us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:31:14.844494 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:31:14.844510 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:31:14.844566 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:31:14.848138 (+ 3572us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:31:14.849069 (+ 931us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:31:14.849078 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:31:14.849080 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:31:14.849272 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:31:14.849799 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:31:14.849803 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:31:14.849805 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:31:14.850324 (+ 519us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:31:14.850584 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0810 11:31:14.850938 (+ 354us) server_negotiation.cc:299] Negotiation successful
0810 11:31:14.851189 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260810 11:31:14.852205 16430 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:31:15.003129 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 16294
I20260810 11:31:15.035673 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 16440
Aug 10 11:31:15 dist-test-slave-srgv krb5kdc[16029](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361473, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.8.18.195@KRBTEST.COM
I20260810 11:31:15.053730 16702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:15.036507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56748 (local address 127.8.18.195:35295)
0810 11:31:15.036776 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:15.036781 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:15.036920 (+ 139us) server_negotiation.cc:407] Connection header received
0810 11:31:15.037092 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:15.037096 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:15.037162 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:15.037242 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:31:15.039019 (+ 1777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:15.039753 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:15.041811 (+ 2058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:15.042250 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:15.042355 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:15.047112 (+ 4757us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:31:15.047129 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:31:15.047145 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:31:15.047197 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:31:15.050513 (+ 3316us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:31:15.051398 (+ 885us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:31:15.051408 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:31:15.051410 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:31:15.051564 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:31:15.052015 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:31:15.052019 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:31:15.052021 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:31:15.052422 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:31:15.052791 (+ 369us) server_negotiation.cc:1035] Waiting for connection context
0810 11:31:15.053117 (+ 326us) server_negotiation.cc:299] Negotiation successful
0810 11:31:15.053367 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260810 11:31:15.054464 16576 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:31:15.183907 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 16440
I20260810 11:31:15.208156 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 16042
I20260810 11:31:15.209620 16117 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:31:15.332432 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 16042
2026-08-10T11:31:15Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (6241 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786361401922362-8267-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 11:31:15 dist-test-slave-srgv krb5kdc[16716](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 11:31:15 dist-test-slave-srgv krb5kdc[16716](info): set up 2 sockets
Aug 10 11:31:15 dist-test-slave-srgv krb5kdc[16716](info): commencing operation
krb5kdc: starting...
W20260810 11:31:16.489575 8267 mini_kdc.cc:121] Time spent starting KDC: real 1.105s user 0.007s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786361401922362-8267-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786361401922362-8267-0/krb5kdc/test-user.keytab.
Aug 10 11:31:16 dist-test-slave-srgv krb5kdc[16716](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361476, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T11:31:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T11:31:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.18.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.18.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.18.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.254.keytab.
Entry for principal kudu/127.8.18.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.254.keytab.
I20260810 11:31:16.759158 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786361401922362-8267-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786361401922362-8267-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786361401922362-8267-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.18.254:37579
--webserver_interface=127.8.18.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.254.keytab
--principal=kudu/127.8.18.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.18.212:35247
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.18.254:37579
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786361401922362-8267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786361401922362-8267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786361401922362-8267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:31:17.136174 16729 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:31:17.136579 16729 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:31:17.136751 16729 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:31:17.144538 16729 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 11:31:17.144742 16729 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:31:17.144832 16729 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:31:17.144922 16729 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 11:31:17.144999 16729 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 11:31:17.155730 16729 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.18.212:35247
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786361401922362-8267-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786361401922362-8267-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.18.254:37579
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.254.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.18.254:37579
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.18.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786361401922362-8267-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:31:17.157248 16729 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:31:17.158735 16729 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:31:17.171634 16734 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:31:17.171631 16735 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:31:17.172454 16737 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:31:17.172968 16729 server_base.cc:1034] running on GCE node
I20260810 11:31:17.173975 16729 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:31:17.175740 16729 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:31:17.177014 16729 hybrid_clock.cc:648] HybridClock initialized: now 1786361477176927 us; error 62 us; skew 500 ppm
I20260810 11:31:17.177536 16729 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-10T11:31:17Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1919 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 11:31:17 dist-test-slave-srgv krb5kdc[16743](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 11:31:17 dist-test-slave-srgv krb5kdc[16743](info): set up 2 sockets
Aug 10 11:31:17 dist-test-slave-srgv krb5kdc[16743](info): commencing operation
krb5kdc: starting...
W20260810 11:31:18.410174 8267 mini_kdc.cc:121] Time spent starting KDC: real 1.106s user 0.000s sys 0.034s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/krb5kdc/test-user.keytab.
Aug 10 11:31:18 dist-test-slave-srgv krb5kdc[16743](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361478, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T11:31:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T11:31:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.18.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.18.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.18.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.254.keytab.
Entry for principal kudu/127.8.18.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.254.keytab.
I20260810 11:31:18.677724 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.18.254:35349
--webserver_interface=127.8.18.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.254.keytab
--principal=kudu/127.8.18.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.18.212:42785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.18.254:35349
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:31:19.040632 16757 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:31:19.040980 16757 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:31:19.041087 16757 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:31:19.048974 16757 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 11:31:19.049135 16757 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:31:19.049185 16757 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:31:19.049250 16757 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 11:31:19.049290 16757 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 11:31:19.060134 16757 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.18.212:42785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.18.254:35349
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.254.keytab
--principal=kudu/127.8.18.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.18.254:35349
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.18.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:31:19.061599 16757 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:31:19.062979 16757 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:31:19.076866 16765 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:31:19.076798 16763 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:31:19.076838 16762 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:31:19.077826 16757 server_base.cc:1034] running on GCE node
I20260810 11:31:19.078689 16757 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:31:19.080324 16757 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:31:19.081559 16757 hybrid_clock.cc:648] HybridClock initialized: now 1786361479081497 us; error 40 us; skew 500 ppm
I20260810 11:31:19.082005 16757 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:31:19 dist-test-slave-srgv krb5kdc[16743](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361479, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:31:19.086027 16757 init.cc:377] Logged in from keytab as kudu/127.8.18.254@KRBTEST.COM (short username kudu)
I20260810 11:31:19.087880 16757 webserver.cc:460] Webserver started at http://127.8.18.254:44543/ using document root <none> and password file <none>
I20260810 11:31:19.088716 16757 fs_manager.cc:359] Metadata directory not provided
I20260810 11:31:19.088845 16757 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:31:19.089221 16757 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:31:19.092787 16757 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/minicluster-data/master-0/data/instance:
uuid: "c5615df8142f423cbfcd9210043ca462"
format_stamp: "Formatted at 2026-08-10 11:31:19 on dist-test-slave-srgv"
server_key: "e150dafaed22caf4a2e2387e52a25970"
server_key_iv: "58bc7e01d9e23fe5b7893e3b58d38e6b"
server_key_version: "encryptionkey@0"
I20260810 11:31:19.093915 16757 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/minicluster-data/master-0/wal/instance:
uuid: "c5615df8142f423cbfcd9210043ca462"
format_stamp: "Formatted at 2026-08-10 11:31:19 on dist-test-slave-srgv"
server_key: "e150dafaed22caf4a2e2387e52a25970"
server_key_iv: "58bc7e01d9e23fe5b7893e3b58d38e6b"
server_key_version: "encryptionkey@0"
I20260810 11:31:19.100263 16757 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260810 11:31:19.104890 16773 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:31:19.106639 16757 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260810 11:31:19.106907 16757 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/minicluster-data/master-0/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/minicluster-data/master-0/wal
uuid: "c5615df8142f423cbfcd9210043ca462"
format_stamp: "Formatted at 2026-08-10 11:31:19 on dist-test-slave-srgv"
server_key: "e150dafaed22caf4a2e2387e52a25970"
server_key_iv: "58bc7e01d9e23fe5b7893e3b58d38e6b"
server_key_version: "encryptionkey@0"
I20260810 11:31:19.107143 16757 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:31:19.122574 16757 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:31:19.128283 16757 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:31:19.128638 16757 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:31:19.150209 16757 rpc_server.cc:307] RPC server started. Bound to: 127.8.18.254:35349
I20260810 11:31:19.150309 16825 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.18.254:35349 every 8 connection(s)
I20260810 11:31:19.151796 16757 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
I20260810 11:31:19.155300 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 16757
I20260810 11:31:19.155586 8267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/minicluster-data/master-0/wal/instance
I20260810 11:31:19.156281 16826 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:31:19.156499 8267 external_mini_cluster.cc:1428] Setting key cb7af0d0c708e0de88c812547888735a
I20260810 11:31:19.170717 16826 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c5615df8142f423cbfcd9210043ca462: Bootstrap starting.
Aug 10 11:31:19 dist-test-slave-srgv krb5kdc[16743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361478, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.18.254@KRBTEST.COM
I20260810 11:31:19.174391 16826 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c5615df8142f423cbfcd9210043ca462: Neither blocks nor log segments found. Creating new log.
I20260810 11:31:19.175719 16826 log.cc:824] T 00000000000000000000000000000000 P c5615df8142f423cbfcd9210043ca462: Log is configured to *not* fsync() on all Append() calls
I20260810 11:31:19.178946 16826 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c5615df8142f423cbfcd9210043ca462: No bootstrap required, opened a new log
I20260810 11:31:19.181560 16829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:19.158895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53596 (local address 127.8.18.254:35349)
0810 11:31:19.159804 (+ 909us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:19.159819 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:19.159872 (+ 53us) server_negotiation.cc:407] Connection header received
0810 11:31:19.161962 (+ 2090us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:19.161991 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:19.162522 (+ 531us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:19.163039 (+ 517us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:31:19.164823 (+ 1784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:19.166318 (+ 1495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:19.167723 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:19.168525 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:19.168737 (+ 212us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:19.173038 (+ 4301us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:31:19.173054 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:31:19.173072 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:31:19.173120 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:31:19.176593 (+ 3473us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:31:19.177464 (+ 871us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:31:19.177473 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:31:19.177479 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:31:19.177664 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:31:19.178103 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:31:19.178107 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:31:19.178110 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:31:19.178673 (+ 563us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:31:19.179012 (+ 339us) server_negotiation.cc:1035] Waiting for connection context
0810 11:31:19.179509 (+ 497us) server_negotiation.cc:299] Negotiation successful
0810 11:31:19.180047 (+ 538us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":685,"thread_start_us":409,"threads_started":1}
I20260810 11:31:19.188671 16826 raft_consensus.cc:348] T 00000000000000000000000000000000 P c5615df8142f423cbfcd9210043ca462 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5615df8142f423cbfcd9210043ca462" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 35349 } }
I20260810 11:31:19.189102 16826 raft_consensus.cc:374] T 00000000000000000000000000000000 P c5615df8142f423cbfcd9210043ca462 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:31:19.189280 16826 raft_consensus.cc:729] T 00000000000000000000000000000000 P c5615df8142f423cbfcd9210043ca462 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5615df8142f423cbfcd9210043ca462, State: Initialized, Role: FOLLOWER
I20260810 11:31:19.189978 16826 consensus_queue.cc:260] T 00000000000000000000000000000000 P c5615df8142f423cbfcd9210043ca462 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5615df8142f423cbfcd9210043ca462" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 35349 } }
I20260810 11:31:19.190311 16826 raft_consensus.cc:388] T 00000000000000000000000000000000 P c5615df8142f423cbfcd9210043ca462 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 11:31:19.190508 16826 raft_consensus.cc:482] T 00000000000000000000000000000000 P c5615df8142f423cbfcd9210043ca462 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 11:31:19.190744 16826 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c5615df8142f423cbfcd9210043ca462 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:31:19.193221 16826 raft_consensus.cc:504] T 00000000000000000000000000000000 P c5615df8142f423cbfcd9210043ca462 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5615df8142f423cbfcd9210043ca462" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 35349 } }
I20260810 11:31:19.193852 16826 leader_election.cc:302] T 00000000000000000000000000000000 P c5615df8142f423cbfcd9210043ca462 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c5615df8142f423cbfcd9210043ca462; no voters:
I20260810 11:31:19.194481 16826 leader_election.cc:288] T 00000000000000000000000000000000 P c5615df8142f423cbfcd9210043ca462 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 11:31:19.194658 16831 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c5615df8142f423cbfcd9210043ca462 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:31:19.195456 16831 raft_consensus.cc:686] T 00000000000000000000000000000000 P c5615df8142f423cbfcd9210043ca462 [term 1 LEADER]: Becoming Leader. State: Replica: c5615df8142f423cbfcd9210043ca462, State: Running, Role: LEADER
I20260810 11:31:19.195997 16831 consensus_queue.cc:237] T 00000000000000000000000000000000 P c5615df8142f423cbfcd9210043ca462 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5615df8142f423cbfcd9210043ca462" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 35349 } }
I20260810 11:31:19.196957 16826 sys_catalog.cc:564] T 00000000000000000000000000000000 P c5615df8142f423cbfcd9210043ca462 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 11:31:19.199200 16833 sys_catalog.cc:455] T 00000000000000000000000000000000 P c5615df8142f423cbfcd9210043ca462 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c5615df8142f423cbfcd9210043ca462" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5615df8142f423cbfcd9210043ca462" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 35349 } } }
I20260810 11:31:19.199666 16833 sys_catalog.cc:458] T 00000000000000000000000000000000 P c5615df8142f423cbfcd9210043ca462 [sys.catalog]: This master's current role is: LEADER
I20260810 11:31:19.199170 16834 sys_catalog.cc:455] T 00000000000000000000000000000000 P c5615df8142f423cbfcd9210043ca462 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c5615df8142f423cbfcd9210043ca462. Latest consensus state: current_term: 1 leader_uuid: "c5615df8142f423cbfcd9210043ca462" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5615df8142f423cbfcd9210043ca462" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 35349 } } }
I20260810 11:31:19.200206 16834 sys_catalog.cc:458] T 00000000000000000000000000000000 P c5615df8142f423cbfcd9210043ca462 [sys.catalog]: This master's current role is: LEADER
I20260810 11:31:19.204700 16838 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 11:31:19.209600 16838 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 11:31:19.219683 16838 catalog_manager.cc:1347] Generated new cluster ID: 1d7ca82cebda4252b67c05e3ff8b773d
I20260810 11:31:19.219830 16838 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 11:31:19.231247 16838 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 11:31:19.232717 16838 catalog_manager.cc:1504] Loading token signing keys...
I20260810 11:31:19.240885 16838 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c5615df8142f423cbfcd9210043ca462: Generated new TSK 0
I20260810 11:31:19.241912 16838 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.18.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.18.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.18.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.193.keytab.
Entry for principal kudu/127.8.18.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.193.keytab.
I20260810 11:31:19.317674 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.18.193:0
--local_ip_for_outbound_sockets=127.8.18.193
--webserver_interface=127.8.18.193
--webserver_port=0
--tserver_master_addrs=127.8.18.254:35349
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.193.keytab
--principal=kudu/127.8.18.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.18.212:42785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:31:19.714715 16854 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:31:19.715052 16854 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:31:19.715155 16854 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:31:19.722956 16854 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:31:19.723101 16854 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:31:19.723220 16854 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.18.193
I20260810 11:31:19.733580 16854 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.18.212:42785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.193.keytab
--principal=kudu/127.8.18.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.18.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.18.193
--webserver_port=0
--tserver_master_addrs=127.8.18.254:35349
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.18.193
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:31:19.735047 16854 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:31:19.736454 16854 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:31:19.750738 16862 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:31:19.750830 16860 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:31:19.750861 16859 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:31:19.751778 16854 server_base.cc:1034] running on GCE node
I20260810 11:31:19.752504 16854 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:31:19.753615 16854 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:31:19.754856 16854 hybrid_clock.cc:648] HybridClock initialized: now 1786361479754803 us; error 44 us; skew 500 ppm
I20260810 11:31:19.755211 16854 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:31:19 dist-test-slave-srgv krb5kdc[16743](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361479, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:31:19.758889 16854 init.cc:377] Logged in from keytab as kudu/127.8.18.193@KRBTEST.COM (short username kudu)
I20260810 11:31:19.760635 16854 webserver.cc:460] Webserver started at http://127.8.18.193:44789/ using document root <none> and password file <none>
I20260810 11:31:19.761475 16854 fs_manager.cc:359] Metadata directory not provided
I20260810 11:31:19.761638 16854 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:31:19.761970 16854 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:31:19.765236 16854 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/minicluster-data/ts-0/data/instance:
uuid: "1a675a692aaf46518e47ad59514383ef"
format_stamp: "Formatted at 2026-08-10 11:31:19 on dist-test-slave-srgv"
server_key: "a7c28e036dcb27ff83261a47143596a2"
server_key_iv: "2509dc64db49e3071d7e37a5b7612338"
server_key_version: "encryptionkey@0"
I20260810 11:31:19.766289 16854 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/minicluster-data/ts-0/wal/instance:
uuid: "1a675a692aaf46518e47ad59514383ef"
format_stamp: "Formatted at 2026-08-10 11:31:19 on dist-test-slave-srgv"
server_key: "a7c28e036dcb27ff83261a47143596a2"
server_key_iv: "2509dc64db49e3071d7e37a5b7612338"
server_key_version: "encryptionkey@0"
I20260810 11:31:19.772322 16854 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.004s
I20260810 11:31:19.776400 16869 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:31:19.778079 16854 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 11:31:19.778306 16854 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/minicluster-data/ts-0/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/minicluster-data/ts-0/wal
uuid: "1a675a692aaf46518e47ad59514383ef"
format_stamp: "Formatted at 2026-08-10 11:31:19 on dist-test-slave-srgv"
server_key: "a7c28e036dcb27ff83261a47143596a2"
server_key_iv: "2509dc64db49e3071d7e37a5b7612338"
server_key_version: "encryptionkey@0"
I20260810 11:31:19.778558 16854 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:31:19.791077 16854 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:31:19.795953 16854 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:31:19.796273 16854 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:31:19.797468 16854 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:31:19.799049 16854 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:31:19.799156 16854 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:31:19.799300 16854 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:31:19.799371 16854 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:31:19.835294 16854 rpc_server.cc:307] RPC server started. Bound to: 127.8.18.193:36787
I20260810 11:31:19.835364 16986 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.18.193:36787 every 8 connection(s)
I20260810 11:31:19.836683 16854 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/minicluster-data/ts-0/data/info.pb
I20260810 11:31:19.841379 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 16854
I20260810 11:31:19.841686 8267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/minicluster-data/ts-0/wal/instance
I20260810 11:31:19.842366 8267 external_mini_cluster.cc:1428] Setting key 8de8a42947e10dd5a90c306d3e1fbc88
Aug 10 11:31:19 dist-test-slave-srgv krb5kdc[16743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361479, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.193@KRBTEST.COM for kudu/127.8.18.254@KRBTEST.COM
I20260810 11:31:19.859251 16989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:19.839464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.193:50357 (local address 127.8.18.254:35349)
0810 11:31:19.839963 (+ 499us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:19.839967 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:19.840987 (+ 1020us) server_negotiation.cc:407] Connection header received
0810 11:31:19.843226 (+ 2239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:19.843232 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:19.843351 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:19.843485 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:31:19.845264 (+ 1779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:19.846111 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:19.847925 (+ 1814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:19.848355 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:19.848548 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:19.852812 (+ 4264us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:31:19.852817 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:31:19.852820 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:31:19.852860 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:31:19.855661 (+ 2801us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:31:19.856578 (+ 917us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:31:19.856589 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:31:19.856591 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:31:19.856738 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:31:19.857216 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:31:19.857224 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:31:19.857227 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:31:19.857610 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:31:19.857785 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0810 11:31:19.858575 (+ 790us) server_negotiation.cc:299] Negotiation successful
0810 11:31:19.858861 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":391,"thread_start_us":270,"threads_started":1}
I20260810 11:31:19.860363 16990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:19.839952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:35349 (local address 127.8.18.193:50357)
0810 11:31:19.840786 (+ 834us) negotiation.cc:107] Waiting for socket to connect
0810 11:31:19.840828 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 11:31:19.842900 (+ 2072us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:31:19.843763 (+ 863us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:31:19.843774 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:31:19.844379 (+ 605us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:31:19.845052 (+ 673us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:19.845074 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:19.846331 (+ 1257us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:19.846335 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:19.847724 (+ 1389us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:19.847748 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:19.848661 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:19.848672 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:19.848952 (+ 280us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:19.849797 (+ 845us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:31:19.849823 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:31:19.852498 (+ 2675us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:31:19.855872 (+ 3374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:31:19.855884 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:31:19.855905 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:31:19.856425 (+ 520us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:31:19.856864 (+ 439us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:31:19.856870 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:31:19.856874 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:31:19.857091 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:31:19.857751 (+ 660us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:31:19.857760 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:31:19.858156 (+ 396us) client_negotiation.cc:769] Sending connection context
0810 11:31:19.858461 (+ 305us) client_negotiation.cc:240] Negotiation successful
0810 11:31:19.859076 (+ 615us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":581,"thread_start_us":316,"threads_started":1}
I20260810 11:31:19.862299 16987 heartbeater.cc:347] Connected to a master server at 127.8.18.254:35349
I20260810 11:31:19.862738 16987 heartbeater.cc:464] Registering TS with master...
I20260810 11:31:19.863778 16987 heartbeater.cc:511] Master 127.8.18.254:35349 requested a full tablet report, sending...
I20260810 11:31:19.866879 16790 ts_manager.cc:194] Registered new tserver with Master: 1a675a692aaf46518e47ad59514383ef (127.8.18.193:36787)
WARNING: no policy specified for kudu/127.8.18.194@KRBTEST.COM; defaulting to no policy
I20260810 11:31:19.869313 16790 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.18.193@KRBTEST.COM'} at 127.8.18.193:50357
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.18.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.18.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.194.keytab.
Entry for principal kudu/127.8.18.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.194.keytab.
I20260810 11:31:19.903148 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.18.194:0
--local_ip_for_outbound_sockets=127.8.18.194
--webserver_interface=127.8.18.194
--webserver_port=0
--tserver_master_addrs=127.8.18.254:35349
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.194.keytab
--principal=kudu/127.8.18.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.18.212:42785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:31:20.265872 16993 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:31:20.266220 16993 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:31:20.266414 16993 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:31:20.274674 16993 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:31:20.274828 16993 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:31:20.275041 16993 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.18.194
I20260810 11:31:20.286610 16993 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.18.212:42785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.194.keytab
--principal=kudu/127.8.18.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.18.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.18.194
--webserver_port=0
--tserver_master_addrs=127.8.18.254:35349
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.18.194
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:31:20.288034 16993 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:31:20.289577 16993 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:31:20.303820 16998 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:31:20.303884 16999 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:31:20.303820 17001 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:31:20.305352 16993 server_base.cc:1034] running on GCE node
I20260810 11:31:20.306136 16993 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:31:20.307180 16993 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:31:20.308421 16993 hybrid_clock.cc:648] HybridClock initialized: now 1786361480308353 us; error 68 us; skew 500 ppm
I20260810 11:31:20.308775 16993 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:31:20 dist-test-slave-srgv krb5kdc[16743](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361480, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:31:20.312881 16993 init.cc:377] Logged in from keytab as kudu/127.8.18.194@KRBTEST.COM (short username kudu)
I20260810 11:31:20.314872 16993 webserver.cc:460] Webserver started at http://127.8.18.194:35509/ using document root <none> and password file <none>
I20260810 11:31:20.315634 16993 fs_manager.cc:359] Metadata directory not provided
I20260810 11:31:20.315752 16993 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:31:20.316039 16993 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:31:20.319353 16993 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/minicluster-data/ts-1/data/instance:
uuid: "e7e47862ec37452e82b48e7c1ab4df01"
format_stamp: "Formatted at 2026-08-10 11:31:20 on dist-test-slave-srgv"
server_key: "9bb35643dc91111491aa775ae4a88b60"
server_key_iv: "fa559d210cc0c693190a046a8c87f707"
server_key_version: "encryptionkey@0"
I20260810 11:31:20.320211 16993 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/minicluster-data/ts-1/wal/instance:
uuid: "e7e47862ec37452e82b48e7c1ab4df01"
format_stamp: "Formatted at 2026-08-10 11:31:20 on dist-test-slave-srgv"
server_key: "9bb35643dc91111491aa775ae4a88b60"
server_key_iv: "fa559d210cc0c693190a046a8c87f707"
server_key_version: "encryptionkey@0"
I20260810 11:31:20.326086 16993 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260810 11:31:20.330080 17008 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:31:20.331699 16993 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 11:31:20.331910 16993 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/minicluster-data/ts-1/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/minicluster-data/ts-1/wal
uuid: "e7e47862ec37452e82b48e7c1ab4df01"
format_stamp: "Formatted at 2026-08-10 11:31:20 on dist-test-slave-srgv"
server_key: "9bb35643dc91111491aa775ae4a88b60"
server_key_iv: "fa559d210cc0c693190a046a8c87f707"
server_key_version: "encryptionkey@0"
I20260810 11:31:20.332147 16993 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:31:20.368301 16993 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:31:20.373044 16993 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:31:20.373474 16993 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:31:20.374787 16993 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:31:20.377144 16993 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:31:20.377295 16993 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:31:20.377471 16993 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:31:20.377555 16993 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260810 11:31:20.413700 16993 rpc_server.cc:307] RPC server started. Bound to: 127.8.18.194:44667
I20260810 11:31:20.413745 17125 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.18.194:44667 every 8 connection(s)
I20260810 11:31:20.415196 16993 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/minicluster-data/ts-1/data/info.pb
I20260810 11:31:20.423298 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 16993
I20260810 11:31:20.423537 8267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/minicluster-data/ts-1/wal/instance
I20260810 11:31:20.424124 8267 external_mini_cluster.cc:1428] Setting key b1997c69f6bb3b3ebb805d70ce82a14a
Aug 10 11:31:20 dist-test-slave-srgv krb5kdc[16743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361480, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.194@KRBTEST.COM for kudu/127.8.18.254@KRBTEST.COM
I20260810 11:31:20.441032 17128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:20.418273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.194:57919 (local address 127.8.18.254:35349)
0810 11:31:20.418767 (+ 494us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:20.418771 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:20.419834 (+ 1063us) server_negotiation.cc:407] Connection header received
0810 11:31:20.421820 (+ 1986us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:20.421824 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:20.421901 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:20.421994 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:31:20.424092 (+ 2098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:20.425065 (+ 973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:20.427240 (+ 2175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:20.428106 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:20.428289 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:20.432406 (+ 4117us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:31:20.432417 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:31:20.432425 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:31:20.432484 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:31:20.436914 (+ 4430us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:31:20.438246 (+ 1332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:31:20.438251 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:31:20.438254 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:31:20.438412 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:31:20.438990 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:31:20.438994 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:31:20.438996 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:31:20.439328 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:31:20.439460 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0810 11:31:20.440493 (+ 1033us) server_negotiation.cc:299] Negotiation successful
0810 11:31:20.440757 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":380,"thread_start_us":245,"threads_started":1}
I20260810 11:31:20.442682 17129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:20.418893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:35349 (local address 127.8.18.194:57919)
0810 11:31:20.419648 (+ 755us) negotiation.cc:107] Waiting for socket to connect
0810 11:31:20.419697 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0810 11:31:20.421491 (+ 1794us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:31:20.422223 (+ 732us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:31:20.422260 (+ 37us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:31:20.422871 (+ 611us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:31:20.423793 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:20.423821 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:20.425270 (+ 1449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:20.425276 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:20.427060 (+ 1784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:20.427075 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:20.428349 (+ 1274us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:20.428355 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:20.428574 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:20.429378 (+ 804us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:31:20.429402 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:31:20.432091 (+ 2689us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:31:20.437145 (+ 5054us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:31:20.437159 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:31:20.437181 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:31:20.438050 (+ 869us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:31:20.438548 (+ 498us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:31:20.438556 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:31:20.438562 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:31:20.438843 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:31:20.439472 (+ 629us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:31:20.439483 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:31:20.440025 (+ 542us) client_negotiation.cc:769] Sending connection context
0810 11:31:20.440375 (+ 350us) client_negotiation.cc:240] Negotiation successful
0810 11:31:20.441018 (+ 643us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":514,"thread_start_us":337,"threads_started":1}
I20260810 11:31:20.444540 17126 heartbeater.cc:347] Connected to a master server at 127.8.18.254:35349
I20260810 11:31:20.444895 17126 heartbeater.cc:464] Registering TS with master...
I20260810 11:31:20.445765 17126 heartbeater.cc:511] Master 127.8.18.254:35349 requested a full tablet report, sending...
I20260810 11:31:20.447510 16790 ts_manager.cc:194] Registered new tserver with Master: e7e47862ec37452e82b48e7c1ab4df01 (127.8.18.194:44667)
I20260810 11:31:20.448886 16790 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.18.194@KRBTEST.COM'} at 127.8.18.194:57919
WARNING: no policy specified for kudu/127.8.18.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.18.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.18.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.195.keytab.
Entry for principal kudu/127.8.18.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.195.keytab.
I20260810 11:31:20.489204 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.18.195:0
--local_ip_for_outbound_sockets=127.8.18.195
--webserver_interface=127.8.18.195
--webserver_port=0
--tserver_master_addrs=127.8.18.254:35349
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.195.keytab
--principal=kudu/127.8.18.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.18.212:42785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:31:20.842943 17132 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:31:20.843241 17132 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:31:20.843339 17132 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:31:20.851020 17132 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:31:20.851167 17132 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:31:20.851282 17132 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.18.195
I20260810 11:31:20.861791 17132 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.18.212:42785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.195.keytab
--principal=kudu/127.8.18.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.18.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.18.195
--webserver_port=0
--tserver_master_addrs=127.8.18.254:35349
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.18.195
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:31:20.863271 17132 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:31:20.864722 17132 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 11:31:20.873397 16987 heartbeater.cc:503] Master 127.8.18.254:35349 was elected leader, sending a full tablet report...
W20260810 11:31:20.879829 17140 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:31:20.879791 17138 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:31:20.879896 17137 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:31:20.880617 17132 server_base.cc:1034] running on GCE node
I20260810 11:31:20.881306 17132 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:31:20.882552 17132 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:31:20.883777 17132 hybrid_clock.cc:648] HybridClock initialized: now 1786361480883730 us; error 50 us; skew 500 ppm
I20260810 11:31:20.884202 17132 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:31:20 dist-test-slave-srgv krb5kdc[16743](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361480, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:31:20.887885 17132 init.cc:377] Logged in from keytab as kudu/127.8.18.195@KRBTEST.COM (short username kudu)
I20260810 11:31:20.889724 17132 webserver.cc:460] Webserver started at http://127.8.18.195:37797/ using document root <none> and password file <none>
I20260810 11:31:20.890550 17132 fs_manager.cc:359] Metadata directory not provided
I20260810 11:31:20.890705 17132 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:31:20.891044 17132 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:31:20.894428 17132 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/minicluster-data/ts-2/data/instance:
uuid: "bacecf932c47408183ac2ca611a92ab7"
format_stamp: "Formatted at 2026-08-10 11:31:20 on dist-test-slave-srgv"
server_key: "db587077ddc39e0ca686cef7f2b05111"
server_key_iv: "2e00a5d78d0117bc43e895803a6d29ee"
server_key_version: "encryptionkey@0"
I20260810 11:31:20.895421 17132 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/minicluster-data/ts-2/wal/instance:
uuid: "bacecf932c47408183ac2ca611a92ab7"
format_stamp: "Formatted at 2026-08-10 11:31:20 on dist-test-slave-srgv"
server_key: "db587077ddc39e0ca686cef7f2b05111"
server_key_iv: "2e00a5d78d0117bc43e895803a6d29ee"
server_key_version: "encryptionkey@0"
I20260810 11:31:20.902223 17132 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260810 11:31:20.906406 17147 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:31:20.907945 17132 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 11:31:20.908152 17132 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/minicluster-data/ts-2/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/minicluster-data/ts-2/wal
uuid: "bacecf932c47408183ac2ca611a92ab7"
format_stamp: "Formatted at 2026-08-10 11:31:20 on dist-test-slave-srgv"
server_key: "db587077ddc39e0ca686cef7f2b05111"
server_key_iv: "2e00a5d78d0117bc43e895803a6d29ee"
server_key_version: "encryptionkey@0"
I20260810 11:31:20.908388 17132 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:31:20.928020 17132 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:31:20.932689 17132 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:31:20.933027 17132 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:31:20.934185 17132 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:31:20.935662 17132 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:31:20.935768 17132 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:31:20.935937 17132 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:31:20.936017 17132 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:31:20.972076 17132 rpc_server.cc:307] RPC server started. Bound to: 127.8.18.195:45471
I20260810 11:31:20.972142 17263 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.18.195:45471 every 8 connection(s)
I20260810 11:31:20.973574 17132 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/minicluster-data/ts-2/data/info.pb
I20260810 11:31:20.977339 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 17132
I20260810 11:31:20.977581 8267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/minicluster-data/ts-2/wal/instance
I20260810 11:31:20.978113 8267 external_mini_cluster.cc:1428] Setting key f1725a5df7e9b4268cace4ddd89a7b3b
Aug 10 11:31:20 dist-test-slave-srgv krb5kdc[16743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361480, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.195@KRBTEST.COM for kudu/127.8.18.254@KRBTEST.COM
I20260810 11:31:20.997551 17266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:20.976310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.195:52531 (local address 127.8.18.254:35349)
0810 11:31:20.976828 (+ 518us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:20.976833 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:20.977898 (+ 1065us) server_negotiation.cc:407] Connection header received
0810 11:31:20.980384 (+ 2486us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:20.980390 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:20.980510 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:20.980645 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:31:20.982796 (+ 2151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:20.983656 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:20.985568 (+ 1912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:20.986023 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:20.986181 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:20.990211 (+ 4030us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:31:20.990217 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:31:20.990220 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:31:20.990262 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:31:20.993745 (+ 3483us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:31:20.994925 (+ 1180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:31:20.994931 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:31:20.994934 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:31:20.995088 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:31:20.995619 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:31:20.995623 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:31:20.995625 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:31:20.995946 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:31:20.996123 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0810 11:31:20.996988 (+ 865us) server_negotiation.cc:299] Negotiation successful
0810 11:31:20.997194 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":406,"thread_start_us":279,"threads_started":1}
I20260810 11:31:20.998615 17267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:20.976774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:35349 (local address 127.8.18.195:52531)
0810 11:31:20.977710 (+ 936us) negotiation.cc:107] Waiting for socket to connect
0810 11:31:20.977759 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0810 11:31:20.979934 (+ 2175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:31:20.980965 (+ 1031us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:31:20.980978 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:31:20.981925 (+ 947us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:31:20.982599 (+ 674us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:20.982629 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:20.983870 (+ 1241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:20.983874 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:20.985332 (+ 1458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:20.985348 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:20.986315 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:20.986321 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:20.986544 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:20.987259 (+ 715us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:31:20.987292 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:31:20.989900 (+ 2608us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:31:20.993952 (+ 4052us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:31:20.993964 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:31:20.993986 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:31:20.994763 (+ 777us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:31:20.995228 (+ 465us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:31:20.995233 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:31:20.995235 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:31:20.995492 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:31:20.996095 (+ 603us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:31:20.996134 (+ 39us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:31:20.996509 (+ 375us) client_negotiation.cc:769] Sending connection context
0810 11:31:20.996799 (+ 290us) client_negotiation.cc:240] Negotiation successful
0810 11:31:20.997279 (+ 480us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":642,"thread_start_us":407,"threads_started":1}
I20260810 11:31:21.000124 17264 heartbeater.cc:347] Connected to a master server at 127.8.18.254:35349
I20260810 11:31:21.000483 17264 heartbeater.cc:464] Registering TS with master...
I20260810 11:31:21.001302 17264 heartbeater.cc:511] Master 127.8.18.254:35349 requested a full tablet report, sending...
I20260810 11:31:21.003050 16790 ts_manager.cc:194] Registered new tserver with Master: bacecf932c47408183ac2ca611a92ab7 (127.8.18.195:45471)
I20260810 11:31:21.003940 16790 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.18.195@KRBTEST.COM'} at 127.8.18.195:52531
I20260810 11:31:21.007532 8267 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.8.18.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.18.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.18.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/krb5kdc/oryx_127.8.18.193.keytab.
Entry for principal oryx/127.8.18.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/krb5kdc/oryx_127.8.18.193.keytab.
WARNING: no policy specified for oryx/127.8.18.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.18.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.18.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/krb5kdc/oryx_127.8.18.194.keytab.
Entry for principal oryx/127.8.18.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/krb5kdc/oryx_127.8.18.194.keytab.
WARNING: no policy specified for oryx/127.8.18.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.18.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.18.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/krb5kdc/oryx_127.8.18.195.keytab.
Entry for principal oryx/127.8.18.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/krb5kdc/oryx_127.8.18.195.keytab.
I20260810 11:31:21.182925 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 16854
Aug 10 11:31:21 dist-test-slave-srgv krb5kdc[16743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361478, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.18.193@KRBTEST.COM
I20260810 11:31:21.200980 17275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:21.184146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52784 (local address 127.8.18.193:36787)
0810 11:31:21.184797 (+ 651us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:21.184804 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:21.184834 (+ 30us) server_negotiation.cc:407] Connection header received
0810 11:31:21.184981 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:21.184988 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:21.185235 (+ 247us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:21.185385 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:31:21.186909 (+ 1524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:21.188024 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:21.189529 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:21.190134 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:21.190257 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:21.194703 (+ 4446us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:31:21.194718 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:31:21.194735 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:31:21.194776 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:31:21.198091 (+ 3315us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:31:21.198985 (+ 894us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:31:21.198993 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:31:21.198996 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:31:21.199133 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:31:21.199499 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:31:21.199502 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:31:21.199504 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:31:21.199873 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:31:21.200106 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0810 11:31:21.200465 (+ 359us) server_negotiation.cc:299] Negotiation successful
0810 11:31:21.200676 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":486,"thread_start_us":316,"threads_started":1}
I20260810 11:31:21.202447 16982 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:31:21.295531 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 16854
I20260810 11:31:21.317241 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 16993
Aug 10 11:31:21 dist-test-slave-srgv krb5kdc[16743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361478, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.18.194@KRBTEST.COM
I20260810 11:31:21.335326 17277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:21.318458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47182 (local address 127.8.18.194:44667)
0810 11:31:21.319077 (+ 619us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:21.319085 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:21.319113 (+ 28us) server_negotiation.cc:407] Connection header received
0810 11:31:21.319176 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:21.319183 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:21.319409 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:21.319550 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:31:21.320955 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:21.322120 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:21.323504 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:21.324098 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:21.324211 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:21.329299 (+ 5088us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:31:21.329315 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:31:21.329332 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:31:21.329381 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:31:21.332391 (+ 3010us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:31:21.333223 (+ 832us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:31:21.333256 (+ 33us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:31:21.333258 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:31:21.333398 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:31:21.333833 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:31:21.333836 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:31:21.333838 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:31:21.334202 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:31:21.334454 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0810 11:31:21.334767 (+ 313us) server_negotiation.cc:299] Negotiation successful
0810 11:31:21.334983 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":455,"thread_start_us":335,"threads_started":1}
I20260810 11:31:21.336824 17121 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:31:21.424397 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 16993
I20260810 11:31:21.443295 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 17132
Aug 10 11:31:21 dist-test-slave-srgv krb5kdc[16743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361478, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.18.195@KRBTEST.COM
I20260810 11:31:21.461716 17280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:21.444451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49940 (local address 127.8.18.195:45471)
0810 11:31:21.445007 (+ 556us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:21.445016 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:21.445048 (+ 32us) server_negotiation.cc:407] Connection header received
0810 11:31:21.445113 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:21.445119 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:21.445349 (+ 230us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:21.445544 (+ 195us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:31:21.446966 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:21.448157 (+ 1191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:21.449622 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:21.450232 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:21.450349 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:21.455264 (+ 4915us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:31:21.455287 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:31:21.455304 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:31:21.455348 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:31:21.458569 (+ 3221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:31:21.459460 (+ 891us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:31:21.459469 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:31:21.459471 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:31:21.459619 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:31:21.460081 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:31:21.460086 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:31:21.460088 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:31:21.460493 (+ 405us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:31:21.460742 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0810 11:31:21.461042 (+ 300us) server_negotiation.cc:299] Negotiation successful
0810 11:31:21.461297 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":405,"thread_start_us":241,"threads_started":1}
I20260810 11:31:21.463122 17259 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:31:21.550441 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 17132
I20260810 11:31:21.570886 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 16757
I20260810 11:31:21.572315 16821 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:31:21.688220 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 16757
I20260810 11:31:21.715369 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.18.254:35349
--webserver_interface=127.8.18.254
--webserver_port=44543
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.254.keytab
--principal=kudu/127.8.18.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.18.212:42785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.18.254:35349
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:31:22.134310 17283 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:31:22.134759 17283 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:31:22.134861 17283 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:31:22.142699 17283 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 11:31:22.142843 17283 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:31:22.142892 17283 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:31:22.142944 17283 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 11:31:22.142983 17283 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 11:31:22.155275 17283 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.18.212:42785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.18.254:35349
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.254.keytab
--principal=kudu/127.8.18.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.18.254:35349
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.18.254
--webserver_port=44543
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:31:22.156819 17283 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:31:22.158357 17283 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:31:22.174080 17288 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:31:22.174063 17289 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:31:22.174021 17291 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:31:22.175236 17283 server_base.cc:1034] running on GCE node
I20260810 11:31:22.176182 17283 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:31:22.178129 17283 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:31:22.179440 17283 hybrid_clock.cc:648] HybridClock initialized: now 1786361482179387 us; error 43 us; skew 500 ppm
I20260810 11:31:22.179906 17283 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:31:22 dist-test-slave-srgv krb5kdc[16743](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361482, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:31:22.184731 17283 init.cc:377] Logged in from keytab as kudu/127.8.18.254@KRBTEST.COM (short username kudu)
I20260810 11:31:22.187347 17283 webserver.cc:460] Webserver started at http://127.8.18.254:44543/ using document root <none> and password file <none>
I20260810 11:31:22.188314 17283 fs_manager.cc:359] Metadata directory not provided
I20260810 11:31:22.188540 17283 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:31:22.195730 17283 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.005s sys 0.000s
I20260810 11:31:22.199651 17298 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:31:22.201370 17283 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.004s
I20260810 11:31:22.201673 17283 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/minicluster-data/master-0/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/minicluster-data/master-0/wal
uuid: "c5615df8142f423cbfcd9210043ca462"
format_stamp: "Formatted at 2026-08-10 11:31:19 on dist-test-slave-srgv"
server_key: "e150dafaed22caf4a2e2387e52a25970"
server_key_iv: "58bc7e01d9e23fe5b7893e3b58d38e6b"
server_key_version: "encryptionkey@0"
I20260810 11:31:22.202879 17283 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:31:22.230883 17283 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:31:22.235992 17283 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:31:22.236338 17283 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:31:22.258312 17283 rpc_server.cc:307] RPC server started. Bound to: 127.8.18.254:35349
I20260810 11:31:22.258319 17351 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.18.254:35349 every 8 connection(s)
I20260810 11:31:22.260061 17283 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
I20260810 11:31:22.264380 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 17283
I20260810 11:31:22.265399 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.18.193:36787
--local_ip_for_outbound_sockets=127.8.18.193
--tserver_master_addrs=127.8.18.254:35349
--webserver_port=44789
--webserver_interface=127.8.18.193
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.193.keytab
--principal=kudu/127.8.18.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.18.212:42785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.8.18.193
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/krb5kdc/oryx_127.8.18.193.keytab with env {KRB5CCNAME=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 11:31:22.268239 17352 sys_catalog.cc:263] Verifying existing consensus state
I20260810 11:31:22.272023 17352 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c5615df8142f423cbfcd9210043ca462: Bootstrap starting.
I20260810 11:31:22.307299 17352 log.cc:824] T 00000000000000000000000000000000 P c5615df8142f423cbfcd9210043ca462: Log is configured to *not* fsync() on all Append() calls
I20260810 11:31:22.323589 17352 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c5615df8142f423cbfcd9210043ca462: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 11:31:22.324391 17352 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c5615df8142f423cbfcd9210043ca462: Bootstrap complete.
I20260810 11:31:22.339275 17352 raft_consensus.cc:348] T 00000000000000000000000000000000 P c5615df8142f423cbfcd9210043ca462 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5615df8142f423cbfcd9210043ca462" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 35349 } }
I20260810 11:31:22.340862 17352 raft_consensus.cc:729] T 00000000000000000000000000000000 P c5615df8142f423cbfcd9210043ca462 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5615df8142f423cbfcd9210043ca462, State: Initialized, Role: FOLLOWER
I20260810 11:31:22.341922 17352 consensus_queue.cc:260] T 00000000000000000000000000000000 P c5615df8142f423cbfcd9210043ca462 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5615df8142f423cbfcd9210043ca462" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 35349 } }
I20260810 11:31:22.342423 17352 raft_consensus.cc:388] T 00000000000000000000000000000000 P c5615df8142f423cbfcd9210043ca462 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 11:31:22.342609 17352 raft_consensus.cc:482] T 00000000000000000000000000000000 P c5615df8142f423cbfcd9210043ca462 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 11:31:22.342873 17352 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c5615df8142f423cbfcd9210043ca462 [term 1 FOLLOWER]: Advancing to term 2
I20260810 11:31:22.350965 17352 raft_consensus.cc:504] T 00000000000000000000000000000000 P c5615df8142f423cbfcd9210043ca462 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5615df8142f423cbfcd9210043ca462" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 35349 } }
I20260810 11:31:22.351917 17352 leader_election.cc:302] T 00000000000000000000000000000000 P c5615df8142f423cbfcd9210043ca462 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c5615df8142f423cbfcd9210043ca462; no voters:
I20260810 11:31:22.352843 17352 leader_election.cc:288] T 00000000000000000000000000000000 P c5615df8142f423cbfcd9210043ca462 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260810 11:31:22.353016 17356 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c5615df8142f423cbfcd9210043ca462 [term 2 FOLLOWER]: Leader election won for term 2
I20260810 11:31:22.354982 17356 raft_consensus.cc:686] T 00000000000000000000000000000000 P c5615df8142f423cbfcd9210043ca462 [term 2 LEADER]: Becoming Leader. State: Replica: c5615df8142f423cbfcd9210043ca462, State: Running, Role: LEADER
I20260810 11:31:22.355711 17356 consensus_queue.cc:237] T 00000000000000000000000000000000 P c5615df8142f423cbfcd9210043ca462 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5615df8142f423cbfcd9210043ca462" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 35349 } }
I20260810 11:31:22.355926 17352 sys_catalog.cc:564] T 00000000000000000000000000000000 P c5615df8142f423cbfcd9210043ca462 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 11:31:22.358572 17358 sys_catalog.cc:455] T 00000000000000000000000000000000 P c5615df8142f423cbfcd9210043ca462 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c5615df8142f423cbfcd9210043ca462. Latest consensus state: current_term: 2 leader_uuid: "c5615df8142f423cbfcd9210043ca462" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5615df8142f423cbfcd9210043ca462" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 35349 } } }
I20260810 11:31:22.359328 17358 sys_catalog.cc:458] T 00000000000000000000000000000000 P c5615df8142f423cbfcd9210043ca462 [sys.catalog]: This master's current role is: LEADER
I20260810 11:31:22.374642 17357 sys_catalog.cc:455] T 00000000000000000000000000000000 P c5615df8142f423cbfcd9210043ca462 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "c5615df8142f423cbfcd9210043ca462" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5615df8142f423cbfcd9210043ca462" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 35349 } } }
I20260810 11:31:22.375321 17361 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 11:31:22.375360 17357 sys_catalog.cc:458] T 00000000000000000000000000000000 P c5615df8142f423cbfcd9210043ca462 [sys.catalog]: This master's current role is: LEADER
I20260810 11:31:22.391901 17361 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 11:31:22.393978 17361 catalog_manager.cc:1259] Loaded cluster ID: 1d7ca82cebda4252b67c05e3ff8b773d
I20260810 11:31:22.394224 17361 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 11:31:22.399250 17361 catalog_manager.cc:1504] Loading token signing keys...
I20260810 11:31:22.400627 17374 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260810 11:31:22.401906 17361 catalog_manager.cc:5879] T 00000000000000000000000000000000 P c5615df8142f423cbfcd9210043ca462: Loaded TSK: 0
I20260810 11:31:22.403800 17361 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260810 11:31:22.743172 17354 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:31:22.743566 17354 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:31:22.743762 17354 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:31:22.752137 17354 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:31:22.752337 17354 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:31:22.752569 17354 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.18.193
I20260810 11:31:22.765318 17354 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.18.212:42785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/krb5kdc/oryx_127.8.18.193.keytab
--principal=oryx/127.8.18.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.18.193:36787
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.18.193
--webserver_port=44789
--tserver_master_addrs=127.8.18.254:35349
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.18.193
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:31:22.767127 17354 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:31:22.768750 17354 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:31:22.788497 17379 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:31:22.788481 17380 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:31:22.789618 17382 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:31:22.791471 17354 server_base.cc:1034] running on GCE node
I20260810 11:31:22.792277 17354 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:31:22.793484 17354 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:31:22.794750 17354 hybrid_clock.cc:648] HybridClock initialized: now 1786361482794671 us; error 70 us; skew 500 ppm
I20260810 11:31:22.795117 17354 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:31:22 dist-test-slave-srgv krb5kdc[16743](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361482, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.18.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:31:22.804229 17354 init.cc:377] Logged in from keytab as oryx/127.8.18.193@KRBTEST.COM (short username oryx)
I20260810 11:31:22.806483 17354 webserver.cc:460] Webserver started at http://127.8.18.193:44789/ using document root <none> and password file <none>
I20260810 11:31:22.807405 17354 fs_manager.cc:359] Metadata directory not provided
I20260810 11:31:22.807595 17354 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:31:22.815156 17354 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.003s sys 0.001s
I20260810 11:31:22.818727 17390 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:31:22.820693 17354 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260810 11:31:22.820937 17354 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/minicluster-data/ts-0/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/minicluster-data/ts-0/wal
uuid: "1a675a692aaf46518e47ad59514383ef"
format_stamp: "Formatted at 2026-08-10 11:31:19 on dist-test-slave-srgv"
server_key: "a7c28e036dcb27ff83261a47143596a2"
server_key_iv: "2509dc64db49e3071d7e37a5b7612338"
server_key_version: "encryptionkey@0"
I20260810 11:31:22.823467 17354 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:31:22.845307 17354 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:31:22.850557 17354 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:31:22.850934 17354 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:31:22.852145 17354 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:31:22.853788 17354 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:31:22.853885 17354 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:31:22.854054 17354 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:31:22.854163 17354 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:31:22.898998 17354 rpc_server.cc:307] RPC server started. Bound to: 127.8.18.193:36787
I20260810 11:31:22.899078 17504 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.18.193:36787 every 8 connection(s)
I20260810 11:31:22.900784 17354 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/minicluster-data/ts-0/data/info.pb
I20260810 11:31:22.908525 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 17354
I20260810 11:31:22.909381 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.18.194:44667
--local_ip_for_outbound_sockets=127.8.18.194
--tserver_master_addrs=127.8.18.254:35349
--webserver_port=35509
--webserver_interface=127.8.18.194
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.194.keytab
--principal=kudu/127.8.18.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.18.212:42785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.8.18.194
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/krb5kdc/oryx_127.8.18.194.keytab with env {KRB5CCNAME=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 11:31:22 dist-test-slave-srgv krb5kdc[16743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.18.193@KRBTEST.COM for oryx/127.8.18.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 11:31:22 dist-test-slave-srgv krb5kdc[16743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.18.193@KRBTEST.COM for oryx/127.8.18.254@KRBTEST.COM, Server not found in Kerberos database
W20260810 11:31:22.926550 17507 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:31:22.904692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.193:43921 (local address 127.8.18.254:35349)
0810 11:31:22.905661 (+ 969us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:22.905683 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:22.905935 (+ 252us) server_negotiation.cc:407] Connection header received
0810 11:31:22.908681 (+ 2746us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:22.908705 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:22.909272 (+ 567us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:22.909772 (+ 500us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:31:22.911508 (+ 1736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:22.914229 (+ 2721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:22.916455 (+ 2226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:22.917977 (+ 1522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:22.918214 (+ 237us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:22.925580 (+ 7366us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.18.193:43921: BlockingRecv error: recv got EOF from 127.8.18.193:43921 (error 108)
Metrics: {"server-negotiator.queue_time_us":649,"thread_start_us":411,"threads_started":1}
I20260810 11:31:22.926614 17508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:22.904718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:35349 (local address 127.8.18.193:43921)
0810 11:31:22.905744 (+ 1026us) negotiation.cc:107] Waiting for socket to connect
0810 11:31:22.905788 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 11:31:22.907898 (+ 2110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:31:22.909947 (+ 2049us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:31:22.909955 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:31:22.910582 (+ 627us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:31:22.911214 (+ 632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:22.911234 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:22.914712 (+ 3478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:22.914719 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:22.916226 (+ 1507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:22.916242 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:22.918267 (+ 2025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:22.918273 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:22.918559 (+ 286us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:22.919395 (+ 836us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:31:22.919419 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:31:22.925057 (+ 5638us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.18.254:35349: Server oryx/127.8.18.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":711,"thread_start_us":449,"threads_started":1}
W20260810 11:31:22.927565 17505 heartbeater.cc:644] Failed to heartbeat to 127.8.18.254:35349 (0 consecutive failures): Not authorized: Failed to ping master at 127.8.18.254:35349: Client connection negotiation failed: client connection to 127.8.18.254:35349: Server oryx/127.8.18.254@KRBTEST.COM not found in Kerberos database
Aug 10 11:31:22 dist-test-slave-srgv krb5kdc[16743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.18.193@KRBTEST.COM for oryx/127.8.18.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 11:31:22 dist-test-slave-srgv krb5kdc[16743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.18.193@KRBTEST.COM for oryx/127.8.18.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 11:31:22.938943 17508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:22.928672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:35349 (local address 127.8.18.193:57881)
0810 11:31:22.928893 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0810 11:31:22.928919 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0810 11:31:22.929050 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:31:22.929532 (+ 482us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:31:22.929537 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:31:22.929927 (+ 390us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:31:22.930080 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:22.930088 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:22.931169 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:22.931176 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:22.932168 (+ 992us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:22.932215 (+ 47us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:22.933607 (+ 1392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:22.933613 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:22.933779 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:22.934558 (+ 779us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:31:22.934603 (+ 45us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:31:22.938712 (+ 4109us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.18.254:35349: Server oryx/127.8.18.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":63}
W20260810 11:31:22.939410 17507 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:31:22.928792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.193:57881 (local address 127.8.18.254:35349)
0810 11:31:22.929019 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:22.929024 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:22.929045 (+ 21us) server_negotiation.cc:407] Connection header received
0810 11:31:22.929247 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:22.929251 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:22.929319 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:22.929489 (+ 170us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:31:22.930249 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:22.930998 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:22.932424 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:22.933293 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:22.933494 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:22.939074 (+ 5580us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.18.193:57881: BlockingRecv error: recv got EOF from 127.8.18.193:57881 (error 108)
Metrics: {"server-negotiator.queue_time_us":92}
Aug 10 11:31:22 dist-test-slave-srgv krb5kdc[16743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.18.193@KRBTEST.COM for oryx/127.8.18.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 11:31:22 dist-test-slave-srgv krb5kdc[16743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.18.193@KRBTEST.COM for oryx/127.8.18.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 11:31:22.949014 17508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:22.939866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:35349 (local address 127.8.18.193:32849)
0810 11:31:22.940071 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0810 11:31:22.940111 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0810 11:31:22.940237 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:31:22.940607 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:31:22.940612 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:31:22.941042 (+ 430us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:31:22.941208 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:22.941218 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:22.942354 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:22.942361 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:22.943395 (+ 1034us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:22.943410 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:22.944598 (+ 1188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:22.944603 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:22.944781 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:22.945570 (+ 789us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:31:22.945589 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:31:22.948769 (+ 3180us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.18.254:35349: Server oryx/127.8.18.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":61}
W20260810 11:31:22.949446 17507 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:31:22.940035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.193:32849 (local address 127.8.18.254:35349)
0810 11:31:22.940227 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:22.940233 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:22.940253 (+ 20us) server_negotiation.cc:407] Connection header received
0810 11:31:22.940356 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:22.940362 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:22.940457 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:22.940578 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:31:22.941397 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:22.942165 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:22.943607 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:22.944386 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:22.944540 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:22.949127 (+ 4587us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.18.193:32849: BlockingRecv error: recv got EOF from 127.8.18.193:32849 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
W20260810 11:31:22.949450 17505 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260810 11:31:23.314306 17509 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:31:23.314791 17509 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:31:23.314977 17509 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:31:23.324332 17509 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:31:23.324536 17509 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:31:23.324791 17509 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.18.194
I20260810 11:31:23.338348 17509 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.18.212:42785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/krb5kdc/oryx_127.8.18.194.keytab
--principal=oryx/127.8.18.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.18.194:44667
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.18.194
--webserver_port=35509
--tserver_master_addrs=127.8.18.254:35349
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.18.194
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:31:23.340211 17509 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:31:23.343321 17509 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:31:23.361210 17517 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:31:23.361951 17514 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:31:23.361835 17515 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:31:23.364272 17509 server_base.cc:1034] running on GCE node
I20260810 11:31:23.365002 17509 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:31:23.366209 17509 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:31:23.367555 17509 hybrid_clock.cc:648] HybridClock initialized: now 1786361483367480 us; error 69 us; skew 500 ppm
I20260810 11:31:23.367946 17509 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:31:23 dist-test-slave-srgv krb5kdc[16743](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361483, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.18.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:31:23.373989 17509 init.cc:377] Logged in from keytab as oryx/127.8.18.194@KRBTEST.COM (short username oryx)
I20260810 11:31:23.375995 17509 webserver.cc:460] Webserver started at http://127.8.18.194:35509/ using document root <none> and password file <none>
I20260810 11:31:23.376775 17509 fs_manager.cc:359] Metadata directory not provided
I20260810 11:31:23.376981 17509 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:31:23.384145 17509 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.005s sys 0.000s
I20260810 11:31:23.387862 17524 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:31:23.390275 17509 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.000s
I20260810 11:31:23.390543 17509 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/minicluster-data/ts-1/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/minicluster-data/ts-1/wal
uuid: "e7e47862ec37452e82b48e7c1ab4df01"
format_stamp: "Formatted at 2026-08-10 11:31:20 on dist-test-slave-srgv"
server_key: "9bb35643dc91111491aa775ae4a88b60"
server_key_iv: "fa559d210cc0c693190a046a8c87f707"
server_key_version: "encryptionkey@0"
I20260810 11:31:23.391639 17509 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:31:23.417984 17509 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:31:23.422724 17509 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:31:23.423038 17509 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:31:23.424136 17509 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:31:23.425709 17509 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:31:23.425799 17509 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:31:23.425961 17509 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:31:23.426054 17509 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:31:23.465935 17509 rpc_server.cc:307] RPC server started. Bound to: 127.8.18.194:44667
I20260810 11:31:23.466007 17640 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.18.194:44667 every 8 connection(s)
I20260810 11:31:23.467568 17509 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/minicluster-data/ts-1/data/info.pb
I20260810 11:31:23.469050 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 17509
I20260810 11:31:23.469974 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.18.195:45471
--local_ip_for_outbound_sockets=127.8.18.195
--tserver_master_addrs=127.8.18.254:35349
--webserver_port=37797
--webserver_interface=127.8.18.195
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.195.keytab
--principal=kudu/127.8.18.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.18.212:42785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.8.18.195
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/krb5kdc/oryx_127.8.18.195.keytab with env {KRB5CCNAME=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 11:31:23 dist-test-slave-srgv krb5kdc[16743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.18.194@KRBTEST.COM for oryx/127.8.18.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 11:31:23 dist-test-slave-srgv krb5kdc[16743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.18.194@KRBTEST.COM for oryx/127.8.18.254@KRBTEST.COM, Server not found in Kerberos database
W20260810 11:31:23.490651 17643 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:31:23.471139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.194:46957 (local address 127.8.18.254:35349)
0810 11:31:23.471804 (+ 665us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:23.471812 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:23.473214 (+ 1402us) server_negotiation.cc:407] Connection header received
0810 11:31:23.476512 (+ 3298us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:23.476517 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:23.476607 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:23.476716 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:31:23.479427 (+ 2711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:23.480756 (+ 1329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:23.483095 (+ 2339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:23.484133 (+ 1038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:23.484327 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:23.490247 (+ 5920us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.18.194:46957: BlockingRecv error: recv got EOF from 127.8.18.194:46957 (error 108)
Metrics: {"server-negotiator.queue_time_us":471,"thread_start_us":296,"threads_started":1}
I20260810 11:31:23.491046 17644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:23.471705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:35349 (local address 127.8.18.194:46957)
0810 11:31:23.472900 (+ 1195us) negotiation.cc:107] Waiting for socket to connect
0810 11:31:23.472998 (+ 98us) client_negotiation.cc:174] Beginning negotiation
0810 11:31:23.476113 (+ 3115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:31:23.477064 (+ 951us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:31:23.477092 (+ 28us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:31:23.478129 (+ 1037us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:31:23.479102 (+ 973us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:23.479136 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:23.481081 (+ 1945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:23.481086 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:23.482823 (+ 1737us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:23.482841 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:23.484442 (+ 1601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:23.484449 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:23.484693 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:23.485640 (+ 947us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:31:23.485674 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:31:23.489859 (+ 4185us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.18.254:35349: Server oryx/127.8.18.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":733,"thread_start_us":412,"threads_started":1}
W20260810 11:31:23.491863 17641 heartbeater.cc:644] Failed to heartbeat to 127.8.18.254:35349 (0 consecutive failures): Not authorized: Failed to ping master at 127.8.18.254:35349: Client connection negotiation failed: client connection to 127.8.18.254:35349: Server oryx/127.8.18.254@KRBTEST.COM not found in Kerberos database
Aug 10 11:31:23 dist-test-slave-srgv krb5kdc[16743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.18.194@KRBTEST.COM for oryx/127.8.18.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 11:31:23 dist-test-slave-srgv krb5kdc[16743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.18.194@KRBTEST.COM for oryx/127.8.18.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 11:31:23.501142 17644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:23.492511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:35349 (local address 127.8.18.194:33651)
0810 11:31:23.492701 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0810 11:31:23.492729 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0810 11:31:23.492844 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:31:23.493174 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:31:23.493179 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:31:23.493651 (+ 472us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:31:23.493800 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:23.493808 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:23.494838 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:23.494844 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:23.495761 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:23.495773 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:23.496502 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:23.496507 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:23.496692 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:23.497351 (+ 659us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:31:23.497370 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:31:23.500827 (+ 3457us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.18.254:35349: Server oryx/127.8.18.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":52}
W20260810 11:31:23.501673 17643 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:31:23.492633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.194:33651 (local address 127.8.18.254:35349)
0810 11:31:23.492823 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:23.492828 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:23.492850 (+ 22us) server_negotiation.cc:407] Connection header received
0810 11:31:23.492959 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:23.492964 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:23.493039 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:23.493147 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:31:23.493952 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:23.494683 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:23.495907 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:23.496351 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:23.496499 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:23.501077 (+ 4578us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.18.194:33651: BlockingRecv error: recv got EOF from 127.8.18.194:33651 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 10 11:31:23 dist-test-slave-srgv krb5kdc[16743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.18.194@KRBTEST.COM for oryx/127.8.18.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 11:31:23 dist-test-slave-srgv krb5kdc[16743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.18.194@KRBTEST.COM for oryx/127.8.18.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 11:31:23.510363 17644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:23.501901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:35349 (local address 127.8.18.194:48769)
0810 11:31:23.502073 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0810 11:31:23.502093 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 11:31:23.502243 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:31:23.502827 (+ 584us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:31:23.502831 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:31:23.503158 (+ 327us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:31:23.503314 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:23.503322 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:23.504385 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:23.504390 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:23.505306 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:23.505318 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:23.506097 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:23.506102 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:23.506288 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:23.506945 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:31:23.506963 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:31:23.510047 (+ 3084us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.18.254:35349: Server oryx/127.8.18.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":54}
W20260810 11:31:23.510603 17643 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:31:23.502360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.194:48769 (local address 127.8.18.254:35349)
0810 11:31:23.502552 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:23.502556 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:23.502572 (+ 16us) server_negotiation.cc:407] Connection header received
0810 11:31:23.502621 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:23.502624 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:23.502679 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:23.502795 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:31:23.503462 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:23.504241 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:23.505495 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:23.505941 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:23.506084 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:23.510289 (+ 4205us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.18.194:48769: BlockingRecv error: recv got EOF from 127.8.18.194:48769 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
W20260810 11:31:23.510807 17641 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260810 11:31:23.834368 17645 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:31:23.834683 17645 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:31:23.834852 17645 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:31:23.842777 17645 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:31:23.842907 17645 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:31:23.843089 17645 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.18.195
I20260810 11:31:23.853683 17645 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.18.212:42785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/krb5kdc/oryx_127.8.18.195.keytab
--principal=oryx/127.8.18.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.18.195:45471
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.18.195
--webserver_port=37797
--tserver_master_addrs=127.8.18.254:35349
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.18.195
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:31:23.855100 17645 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:31:23.856617 17645 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:31:23.871317 17653 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:31:23.871416 17651 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:31:23.871392 17650 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:31:23.872462 17645 server_base.cc:1034] running on GCE node
I20260810 11:31:23.873241 17645 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:31:23.874346 17645 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:31:23.875557 17645 hybrid_clock.cc:648] HybridClock initialized: now 1786361483875488 us; error 62 us; skew 500 ppm
I20260810 11:31:23.875907 17645 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:31:23 dist-test-slave-srgv krb5kdc[16743](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361483, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.18.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:31:23.879626 17645 init.cc:377] Logged in from keytab as oryx/127.8.18.195@KRBTEST.COM (short username oryx)
I20260810 11:31:23.881351 17645 webserver.cc:460] Webserver started at http://127.8.18.195:37797/ using document root <none> and password file <none>
I20260810 11:31:23.882194 17645 fs_manager.cc:359] Metadata directory not provided
I20260810 11:31:23.882378 17645 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:31:23.888377 17645 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.002s
I20260810 11:31:23.891527 17660 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:31:23.893239 17645 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 11:31:23.893512 17645 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/minicluster-data/ts-2/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/minicluster-data/ts-2/wal
uuid: "bacecf932c47408183ac2ca611a92ab7"
format_stamp: "Formatted at 2026-08-10 11:31:20 on dist-test-slave-srgv"
server_key: "db587077ddc39e0ca686cef7f2b05111"
server_key_iv: "2e00a5d78d0117bc43e895803a6d29ee"
server_key_version: "encryptionkey@0"
I20260810 11:31:23.894609 17645 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:31:23.910703 17645 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:31:23.915076 17645 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:31:23.915323 17645 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:31:23.916412 17645 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:31:23.917927 17645 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:31:23.918030 17645 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:31:23.918115 17645 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:31:23.918171 17645 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 11:31:23 dist-test-slave-srgv krb5kdc[16743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.18.193@KRBTEST.COM for oryx/127.8.18.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 11:31:23 dist-test-slave-srgv krb5kdc[16743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.18.193@KRBTEST.COM for oryx/127.8.18.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 11:31:23.964036 17645 rpc_server.cc:307] RPC server started. Bound to: 127.8.18.195:45471
I20260810 11:31:23.964089 17777 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.18.195:45471 every 8 connection(s)
I20260810 11:31:23.964953 17759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:23.951096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:35349 (local address 127.8.18.193:37739)
0810 11:31:23.952578 (+ 1482us) negotiation.cc:107] Waiting for socket to connect
0810 11:31:23.952598 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 11:31:23.952808 (+ 210us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:31:23.953314 (+ 506us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:31:23.953324 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:31:23.954378 (+ 1054us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:31:23.954758 (+ 380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:23.954779 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:23.956141 (+ 1362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:23.956150 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:23.958166 (+ 2016us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:23.958196 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:23.959882 (+ 1686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:23.959887 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:23.960150 (+ 263us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:23.961042 (+ 892us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:31:23.961076 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:31:23.964688 (+ 3612us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.18.254:35349: Server oryx/127.8.18.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":1356,"thread_start_us":1235,"threads_started":1}
W20260810 11:31:23.965385 17643 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:31:23.951450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.193:37739 (local address 127.8.18.254:35349)
0810 11:31:23.951654 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:23.951661 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:23.952749 (+ 1088us) server_negotiation.cc:407] Connection header received
0810 11:31:23.952963 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:23.952968 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:23.953059 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:23.953173 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:31:23.955062 (+ 1889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:23.955954 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:23.958420 (+ 2466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:23.959318 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:23.959458 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:23.965068 (+ 5610us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.18.193:37739: BlockingRecv error: recv got EOF from 127.8.18.193:37739 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
I20260810 11:31:23.965969 17645 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786361401922362-8267-0/minicluster-data/ts-2/data/info.pb
I20260810 11:31:23.974876 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 17645
Aug 10 11:31:23 dist-test-slave-srgv krb5kdc[16743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.18.195@KRBTEST.COM for oryx/127.8.18.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 11:31:23 dist-test-slave-srgv krb5kdc[16743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.18.195@KRBTEST.COM for oryx/127.8.18.254@KRBTEST.COM, Server not found in Kerberos database
W20260810 11:31:23.985760 17643 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:31:23.968926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.195:58501 (local address 127.8.18.254:35349)
0810 11:31:23.969114 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:23.969119 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:23.970428 (+ 1309us) server_negotiation.cc:407] Connection header received
0810 11:31:23.972483 (+ 2055us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:23.972488 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:23.972565 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:23.972659 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:31:23.974427 (+ 1768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:23.975615 (+ 1188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:23.978306 (+ 2691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:23.978774 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:23.978938 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:23.985544 (+ 6606us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.18.195:58501: BlockingRecv error: recv got EOF from 127.8.18.195:58501 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
I20260810 11:31:23.986963 17780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:23.969313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:35349 (local address 127.8.18.195:58501)
0810 11:31:23.970203 (+ 890us) negotiation.cc:107] Waiting for socket to connect
0810 11:31:23.970284 (+ 81us) client_negotiation.cc:174] Beginning negotiation
0810 11:31:23.972144 (+ 1860us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:31:23.972894 (+ 750us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:31:23.972904 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:31:23.973582 (+ 678us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:31:23.974195 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:23.974213 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:23.975821 (+ 1608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:23.975825 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:23.978099 (+ 2274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:23.978129 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:23.979058 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:23.979066 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:23.979339 (+ 273us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:23.980099 (+ 760us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:31:23.980135 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:31:23.985286 (+ 5151us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.18.254:35349: Server oryx/127.8.18.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":611,"thread_start_us":311,"threads_started":1}
W20260810 11:31:23.987799 17778 heartbeater.cc:644] Failed to heartbeat to 127.8.18.254:35349 (0 consecutive failures): Not authorized: Failed to ping master at 127.8.18.254:35349: Client connection negotiation failed: client connection to 127.8.18.254:35349: Server oryx/127.8.18.254@KRBTEST.COM not found in Kerberos database
Aug 10 11:31:23 dist-test-slave-srgv krb5kdc[16743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.18.195@KRBTEST.COM for oryx/127.8.18.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 11:31:23.996832 17781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:23.976015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53598 (local address 127.8.18.254:35349)
0810 11:31:23.976990 (+ 975us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:23.976994 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:23.977445 (+ 451us) server_negotiation.cc:407] Connection header received
0810 11:31:23.977669 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:23.977677 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:23.977761 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:23.977890 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:31:23.979740 (+ 1850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:23.980944 (+ 1204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:23.982931 (+ 1987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:23.983524 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:23.983634 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:23.987662 (+ 4028us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:31:23.987676 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:31:23.987697 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:31:23.987780 (+ 83us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:31:23.992307 (+ 4527us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:31:23.993348 (+ 1041us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:31:23.993360 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:31:23.993369 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:31:23.993681 (+ 312us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:31:23.994254 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:31:23.994258 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:31:23.994260 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:31:23.994820 (+ 560us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:31:23.995055 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0810 11:31:23.995776 (+ 721us) server_negotiation.cc:299] Negotiation successful
0810 11:31:23.996347 (+ 571us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":846,"thread_start_us":695,"threads_started":1}
Aug 10 11:31:23 dist-test-slave-srgv krb5kdc[16743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.18.195@KRBTEST.COM for oryx/127.8.18.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 11:31:23.998276 17780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:23.988507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:35349 (local address 127.8.18.195:40025)
0810 11:31:23.988721 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0810 11:31:23.988754 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0810 11:31:23.988881 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:31:23.989265 (+ 384us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:31:23.989273 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:31:23.989762 (+ 489us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:31:23.989991 (+ 229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:23.990007 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:23.991098 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:23.991103 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:23.992599 (+ 1496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:23.992613 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:23.993487 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:23.993492 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:23.993627 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:23.994282 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:31:23.994301 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:31:23.997977 (+ 3676us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.18.254:35349: Server oryx/127.8.18.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":66}
W20260810 11:31:23.998564 17643 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:31:23.988624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.195:40025 (local address 127.8.18.254:35349)
0810 11:31:23.988935 (+ 311us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:23.988946 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:23.988968 (+ 22us) server_negotiation.cc:407] Connection header received
0810 11:31:23.989033 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:23.989041 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:23.989119 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:23.989207 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:31:23.990157 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:23.990943 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:23.992750 (+ 1807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:23.993265 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:23.993396 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:23.998259 (+ 4863us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.18.195:40025: BlockingRecv error: recv got EOF from 127.8.18.195:40025 (error 108)
Metrics: {"server-negotiator.queue_time_us":167}
Aug 10 11:31:24 dist-test-slave-srgv krb5kdc[16743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.18.195@KRBTEST.COM for oryx/127.8.18.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 11:31:24 dist-test-slave-srgv krb5kdc[16743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.18.195@KRBTEST.COM for oryx/127.8.18.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 11:31:24.008401 17780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:23.999431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:35349 (local address 127.8.18.195:51147)
0810 11:31:23.999616 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0810 11:31:23.999636 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 11:31:23.999759 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:31:24.000666 (+ 907us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:31:24.000671 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:31:24.001069 (+ 398us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:31:24.001240 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:24.001249 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:24.002425 (+ 1176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:24.002431 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:24.003356 (+ 925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:24.003368 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:24.004125 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:24.004130 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:24.004327 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:24.004912 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:31:24.004929 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:31:24.008033 (+ 3104us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.18.254:35349: Server oryx/127.8.18.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":47}
W20260810 11:31:24.008744 17643 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:31:23.999549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.195:51147 (local address 127.8.18.254:35349)
0810 11:31:23.999710 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:23.999714 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:23.999753 (+ 39us) server_negotiation.cc:407] Connection header received
0810 11:31:23.999921 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:23.999928 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:24.000020 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:24.000652 (+ 632us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:31:24.001396 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:24.002271 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:24.003510 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:24.003952 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:24.004112 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:24.008327 (+ 4215us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.18.195:51147: BlockingRecv error: recv got EOF from 127.8.18.195:51147 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
W20260810 11:31:24.008744 17778 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 10 11:31:24 dist-test-slave-srgv krb5kdc[16743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.18.194@KRBTEST.COM for oryx/127.8.18.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 11:31:24 dist-test-slave-srgv krb5kdc[16743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.18.194@KRBTEST.COM for oryx/127.8.18.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 11:31:24.520533 17783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:24.511536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:35349 (local address 127.8.18.194:47555)
0810 11:31:24.512033 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0810 11:31:24.512056 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0810 11:31:24.512164 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:31:24.512615 (+ 451us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:31:24.512619 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:31:24.513089 (+ 470us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:31:24.513296 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:24.513306 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:24.514415 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:24.514420 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:24.515447 (+ 1027us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:24.515462 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:24.516245 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:24.516250 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:24.516384 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:24.517026 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:31:24.517045 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:31:24.520266 (+ 3221us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.18.254:35349: Server oryx/127.8.18.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":376,"thread_start_us":249,"threads_started":1}
W20260810 11:31:24.520891 17784 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:31:24.511583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.194:47555 (local address 127.8.18.254:35349)
0810 11:31:24.512326 (+ 743us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:24.512331 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:24.512346 (+ 15us) server_negotiation.cc:407] Connection header received
0810 11:31:24.512393 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:24.512397 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:24.512455 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:24.512544 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:31:24.513497 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:24.514235 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:24.515605 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:24.516041 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:24.516139 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:24.520552 (+ 4413us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.18.194:47555: BlockingRecv error: recv got EOF from 127.8.18.194:47555 (error 108)
Metrics: {"server-negotiator.queue_time_us":646,"thread_start_us":545,"threads_started":1}
Aug 10 11:31:24 dist-test-slave-srgv krb5kdc[16743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.18.193@KRBTEST.COM for oryx/127.8.18.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 11:31:24 dist-test-slave-srgv krb5kdc[16743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.18.193@KRBTEST.COM for oryx/127.8.18.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 11:31:24.976357 17785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:24.966090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:35349 (local address 127.8.18.193:57847)
0810 11:31:24.966822 (+ 732us) negotiation.cc:107] Waiting for socket to connect
0810 11:31:24.966841 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 11:31:24.966971 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:31:24.967363 (+ 392us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:31:24.967367 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:31:24.967775 (+ 408us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:31:24.967927 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:24.967936 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:24.969532 (+ 1596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:24.969538 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:24.970662 (+ 1124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:24.970707 (+ 45us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:24.971748 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:24.971754 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:24.971882 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:24.972612 (+ 730us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:31:24.972631 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:31:24.975949 (+ 3318us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.18.254:35349: Server oryx/127.8.18.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":574,"thread_start_us":439,"threads_started":1}
W20260810 11:31:24.976449 17784 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:31:24.966300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.193:57847 (local address 127.8.18.254:35349)
0810 11:31:24.966548 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:24.966555 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:24.966960 (+ 405us) server_negotiation.cc:407] Connection header received
0810 11:31:24.967098 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:24.967104 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:24.967192 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:24.967312 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:31:24.968122 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:24.969280 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:24.970931 (+ 1651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:24.971556 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:24.971678 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:24.976218 (+ 4540us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.18.193:57847: BlockingRecv error: recv got EOF from 127.8.18.193:57847 (error 108)
Metrics: {"server-negotiator.queue_time_us":91}
Aug 10 11:31:25 dist-test-slave-srgv krb5kdc[16743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.18.195@KRBTEST.COM for oryx/127.8.18.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 11:31:25 dist-test-slave-srgv krb5kdc[16743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.18.195@KRBTEST.COM for oryx/127.8.18.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 11:31:25.018093 17786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:25.009482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:35349 (local address 127.8.18.195:41267)
0810 11:31:25.010051 (+ 569us) negotiation.cc:107] Waiting for socket to connect
0810 11:31:25.010067 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 11:31:25.010186 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:31:25.010493 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:31:25.010497 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:31:25.010988 (+ 491us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:31:25.011183 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:25.011193 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:25.012257 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:25.012261 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:25.013244 (+ 983us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:25.013256 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:25.014038 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:25.014042 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:25.014165 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:25.014798 (+ 633us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:31:25.014816 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:31:25.017848 (+ 3032us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.18.254:35349: Server oryx/127.8.18.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":439,"thread_start_us":310,"threads_started":1}
W20260810 11:31:25.018409 17784 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:31:25.009550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.195:41267 (local address 127.8.18.254:35349)
0810 11:31:25.009837 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:25.009843 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:25.010154 (+ 311us) server_negotiation.cc:407] Connection header received
0810 11:31:25.010286 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:25.010290 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:25.010356 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:25.010440 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:31:25.011352 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:25.012124 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:25.013389 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:25.013866 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:25.013965 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:25.018127 (+ 4162us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.18.195:41267: BlockingRecv error: recv got EOF from 127.8.18.195:41267 (error 108)
Metrics: {"server-negotiator.queue_time_us":96}
Aug 10 11:31:25 dist-test-slave-srgv krb5kdc[16743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.18.194@KRBTEST.COM for oryx/127.8.18.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 11:31:25 dist-test-slave-srgv krb5kdc[16743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.18.194@KRBTEST.COM for oryx/127.8.18.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 11:31:25.531112 17787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:25.521811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:35349 (local address 127.8.18.194:58171)
0810 11:31:25.522407 (+ 596us) negotiation.cc:107] Waiting for socket to connect
0810 11:31:25.522430 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0810 11:31:25.522590 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:31:25.523002 (+ 412us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:31:25.523008 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:31:25.523468 (+ 460us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:31:25.523716 (+ 248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:25.523728 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:25.524818 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:25.524823 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:25.525858 (+ 1035us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:25.525872 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:25.526610 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:25.526614 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:25.526730 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:25.527439 (+ 709us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:31:25.527457 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:31:25.530740 (+ 3283us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.18.254:35349: Server oryx/127.8.18.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":461,"thread_start_us":339,"threads_started":1}
W20260810 11:31:25.531356 17788 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:31:25.521865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.194:58171 (local address 127.8.18.254:35349)
0810 11:31:25.522398 (+ 533us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:25.522404 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:25.522541 (+ 137us) server_negotiation.cc:407] Connection header received
0810 11:31:25.522711 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:25.522716 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:25.522819 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:25.522942 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:31:25.523861 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:25.524658 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:25.526010 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:25.526443 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:25.526554 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:25.531050 (+ 4496us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.18.194:58171: BlockingRecv error: recv got EOF from 127.8.18.194:58171 (error 108)
Metrics: {"server-negotiator.queue_time_us":403,"thread_start_us":288,"threads_started":1}
Aug 10 11:31:25 dist-test-slave-srgv krb5kdc[16743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.18.193@KRBTEST.COM for oryx/127.8.18.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 11:31:25 dist-test-slave-srgv krb5kdc[16743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.18.193@KRBTEST.COM for oryx/127.8.18.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 11:31:25.986785 17789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:25.977594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:35349 (local address 127.8.18.193:35023)
0810 11:31:25.978159 (+ 565us) negotiation.cc:107] Waiting for socket to connect
0810 11:31:25.978176 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 11:31:25.978342 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:31:25.978717 (+ 375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:31:25.978722 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:31:25.979151 (+ 429us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:31:25.979360 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:25.979370 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:25.980432 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:25.980437 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:25.981356 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:25.981368 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:25.982140 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:25.982146 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:25.982352 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:25.983036 (+ 684us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:31:25.983098 (+ 62us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:31:25.986364 (+ 3266us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.18.254:35349: Server oryx/127.8.18.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":459,"thread_start_us":322,"threads_started":1}
W20260810 11:31:25.986907 17788 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:31:25.977807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.193:35023 (local address 127.8.18.254:35349)
0810 11:31:25.978046 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:25.978053 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:25.978263 (+ 210us) server_negotiation.cc:407] Connection header received
0810 11:31:25.978483 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:25.978491 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:25.978569 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:25.978680 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:31:25.979515 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:25.980280 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:25.981544 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:25.981972 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:25.982117 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:25.986619 (+ 4502us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.18.193:35023: BlockingRecv error: recv got EOF from 127.8.18.193:35023 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
Aug 10 11:31:26 dist-test-slave-srgv krb5kdc[16743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.18.195@KRBTEST.COM for oryx/127.8.18.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 11:31:26 dist-test-slave-srgv krb5kdc[16743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.18.195@KRBTEST.COM for oryx/127.8.18.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 11:31:26.028908 17790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:26.019275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:35349 (local address 127.8.18.195:36597)
0810 11:31:26.019924 (+ 649us) negotiation.cc:107] Waiting for socket to connect
0810 11:31:26.019941 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 11:31:26.020129 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:31:26.020518 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:31:26.020522 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:31:26.020990 (+ 468us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:31:26.021225 (+ 235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:26.021265 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:26.022361 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:26.022366 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:26.023268 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:26.023280 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:26.024052 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:26.024058 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:26.024240 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:26.024952 (+ 712us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:31:26.024976 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:31:26.028646 (+ 3670us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.18.254:35349: Server oryx/127.8.18.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":534,"thread_start_us":362,"threads_started":1}
W20260810 11:31:26.029162 17788 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:31:26.019446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.195:36597 (local address 127.8.18.254:35349)
0810 11:31:26.019741 (+ 295us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:26.019749 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:26.020051 (+ 302us) server_negotiation.cc:407] Connection header received
0810 11:31:26.020264 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:26.020273 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:26.020369 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:26.020485 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:31:26.021447 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:26.022187 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:26.023429 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:26.023870 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:26.024031 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:26.028876 (+ 4845us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.18.195:36597: BlockingRecv error: recv got EOF from 127.8.18.195:36597 (error 108)
Metrics: {"server-negotiator.queue_time_us":88}
Aug 10 11:31:26 dist-test-slave-srgv krb5kdc[16743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.18.194@KRBTEST.COM for oryx/127.8.18.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 11:31:26 dist-test-slave-srgv krb5kdc[16743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.18.194@KRBTEST.COM for oryx/127.8.18.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 11:31:26.541815 17792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:26.532211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:35349 (local address 127.8.18.194:54599)
0810 11:31:26.532822 (+ 611us) negotiation.cc:107] Waiting for socket to connect
0810 11:31:26.532842 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 11:31:26.532958 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:31:26.533500 (+ 542us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:31:26.533505 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:31:26.533887 (+ 382us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:31:26.534067 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:26.534118 (+ 51us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:26.535362 (+ 1244us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:26.535366 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:26.536330 (+ 964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:26.536344 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:26.537151 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:26.537155 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:26.537295 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:26.538111 (+ 816us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:31:26.538131 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:31:26.541439 (+ 3308us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.18.254:35349: Server oryx/127.8.18.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":491,"thread_start_us":320,"threads_started":1}
W20260810 11:31:26.542356 17793 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:31:26.532276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.194:54599 (local address 127.8.18.254:35349)
0810 11:31:26.532875 (+ 599us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:26.532880 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:26.532899 (+ 19us) server_negotiation.cc:407] Connection header received
0810 11:31:26.533125 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:26.533130 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:26.533240 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:26.533350 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:31:26.534281 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:26.535209 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:26.536494 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:26.536965 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:26.537069 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:26.541797 (+ 4728us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.18.194:54599: BlockingRecv error: recv got EOF from 127.8.18.194:54599 (error 108)
Metrics: {"server-negotiator.queue_time_us":492,"thread_start_us":359,"threads_started":1}
Aug 10 11:31:26 dist-test-slave-srgv krb5kdc[16743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.18.193@KRBTEST.COM for oryx/127.8.18.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 11:31:26 dist-test-slave-srgv krb5kdc[16743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.18.193@KRBTEST.COM for oryx/127.8.18.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 11:31:26.996961 17794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:26.988075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:35349 (local address 127.8.18.193:37377)
0810 11:31:26.988654 (+ 579us) negotiation.cc:107] Waiting for socket to connect
0810 11:31:26.988671 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 11:31:26.988787 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:31:26.989197 (+ 410us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:31:26.989201 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:31:26.989663 (+ 462us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:31:26.989830 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:26.989840 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:26.990887 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:26.990891 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:26.991889 (+ 998us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:26.991904 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:26.992660 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:26.992665 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:26.992821 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:26.993532 (+ 711us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:31:26.993551 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:31:26.996613 (+ 3062us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.18.254:35349: Server oryx/127.8.18.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":453,"thread_start_us":334,"threads_started":1}
W20260810 11:31:26.997131 17793 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:31:26.988177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.193:37377 (local address 127.8.18.254:35349)
0810 11:31:26.988393 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:26.988402 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:26.988753 (+ 351us) server_negotiation.cc:407] Connection header received
0810 11:31:26.988933 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:26.988940 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:26.989033 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:26.989145 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:31:26.989986 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:26.990751 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:26.992043 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:26.992462 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:26.992562 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:26.996881 (+ 4319us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.18.193:37377: BlockingRecv error: recv got EOF from 127.8.18.193:37377 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
Aug 10 11:31:27 dist-test-slave-srgv krb5kdc[16743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.18.195@KRBTEST.COM for oryx/127.8.18.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 11:31:27 dist-test-slave-srgv krb5kdc[16743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.18.195@KRBTEST.COM for oryx/127.8.18.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 11:31:27.039085 17795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:27.029947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:35349 (local address 127.8.18.195:52669)
0810 11:31:27.030578 (+ 631us) negotiation.cc:107] Waiting for socket to connect
0810 11:31:27.030596 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 11:31:27.030731 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:31:27.031061 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:31:27.031065 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:31:27.031417 (+ 352us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:31:27.031643 (+ 226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:27.031652 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:27.032734 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:27.032737 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:27.033747 (+ 1010us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:27.033761 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:27.034464 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:27.034468 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:27.034624 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:27.035243 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:31:27.035260 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:31:27.038739 (+ 3479us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.18.254:35349: Server oryx/127.8.18.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":497,"thread_start_us":383,"threads_started":1}
W20260810 11:31:27.039547 17793 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:31:27.030100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.195:52669 (local address 127.8.18.254:35349)
0810 11:31:27.030324 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:27.030329 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:27.030696 (+ 367us) server_negotiation.cc:407] Connection header received
0810 11:31:27.030853 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:27.030857 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:27.030924 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:27.031013 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:31:27.031793 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:27.032587 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:27.033881 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:27.034323 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:27.034423 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:27.039025 (+ 4602us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.18.195:52669: BlockingRecv error: recv got EOF from 127.8.18.195:52669 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Aug 10 11:31:27 dist-test-slave-srgv krb5kdc[16743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.18.194@KRBTEST.COM for oryx/127.8.18.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 11:31:27 dist-test-slave-srgv krb5kdc[16743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.18.194@KRBTEST.COM for oryx/127.8.18.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 11:31:27.552291 17796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:27.542959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:35349 (local address 127.8.18.194:43753)
0810 11:31:27.543506 (+ 547us) negotiation.cc:107] Waiting for socket to connect
0810 11:31:27.543524 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 11:31:27.543639 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:31:27.543993 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:31:27.543997 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:31:27.544414 (+ 417us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:31:27.544570 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:27.544579 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:27.545837 (+ 1258us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:27.545842 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:27.546818 (+ 976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:27.546831 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:27.547679 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:27.547686 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:27.547884 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:27.548623 (+ 739us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:31:27.548642 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:31:27.551934 (+ 3292us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.18.254:35349: Server oryx/127.8.18.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":423,"thread_start_us":310,"threads_started":1}
W20260810 11:31:27.552466 17797 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:31:27.543075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.194:43753 (local address 127.8.18.254:35349)
0810 11:31:27.543622 (+ 547us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:27.543627 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:27.543654 (+ 27us) server_negotiation.cc:407] Connection header received
0810 11:31:27.543741 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:27.543745 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:27.543823 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:27.543937 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:31:27.544768 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:27.545709 (+ 941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:27.546999 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:27.547510 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:27.547624 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:27.552185 (+ 4561us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.18.194:43753: BlockingRecv error: recv got EOF from 127.8.18.194:43753 (error 108)
Metrics: {"server-negotiator.queue_time_us":442,"thread_start_us":325,"threads_started":1}
Aug 10 11:31:28 dist-test-slave-srgv krb5kdc[16743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.18.193@KRBTEST.COM for oryx/127.8.18.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 11:31:28 dist-test-slave-srgv krb5kdc[16743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.18.193@KRBTEST.COM for oryx/127.8.18.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 11:31:28.013087 17798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:27.998361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:35349 (local address 127.8.18.193:53801)
0810 11:31:27.999140 (+ 779us) negotiation.cc:107] Waiting for socket to connect
0810 11:31:27.999159 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 11:31:27.999304 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:31:27.999735 (+ 431us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:31:27.999739 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:31:28.000295 (+ 556us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:31:28.000472 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:28.000482 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:28.001665 (+ 1183us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:28.001670 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:28.002669 (+ 999us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:28.002683 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:28.003529 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:28.003535 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:28.003730 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:28.009488 (+ 5758us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:31:28.009516 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:31:28.012812 (+ 3296us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.18.254:35349: Server oryx/127.8.18.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":563,"thread_start_us":363,"threads_started":1}
W20260810 11:31:28.013525 17797 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:31:27.998443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.193:53801 (local address 127.8.18.254:35349)
0810 11:31:27.998704 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:27.998715 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:27.999289 (+ 574us) server_negotiation.cc:407] Connection header received
0810 11:31:27.999409 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:27.999414 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:27.999500 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:27.999613 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:31:28.000637 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:28.001519 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:28.002814 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:28.003350 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:28.003492 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:28.013178 (+ 9686us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.18.193:53801: BlockingRecv error: recv got EOF from 127.8.18.193:53801 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
Aug 10 11:31:28 dist-test-slave-srgv krb5kdc[16743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.18.195@KRBTEST.COM for oryx/127.8.18.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 11:31:28 dist-test-slave-srgv krb5kdc[16743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.18.195@KRBTEST.COM for oryx/127.8.18.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 11:31:28.049193 17799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:28.040250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:35349 (local address 127.8.18.195:47213)
0810 11:31:28.040850 (+ 600us) negotiation.cc:107] Waiting for socket to connect
0810 11:31:28.040867 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 11:31:28.041047 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:31:28.041505 (+ 458us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:31:28.041509 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:31:28.041924 (+ 415us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:31:28.042076 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:28.042086 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:28.043166 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:28.043174 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:28.044169 (+ 995us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:28.044183 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:28.044934 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:28.044940 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:28.045150 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:28.045849 (+ 699us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:31:28.045867 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:31:28.048947 (+ 3080us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.18.254:35349: Server oryx/127.8.18.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":498,"thread_start_us":318,"threads_started":1}
W20260810 11:31:28.049568 17797 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:31:28.040398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.195:47213 (local address 127.8.18.254:35349)
0810 11:31:28.040659 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:28.040666 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:28.040980 (+ 314us) server_negotiation.cc:407] Connection header received
0810 11:31:28.041191 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:28.041196 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:28.041310 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:28.041452 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:31:28.042251 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:28.043022 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:28.044327 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:28.044777 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:28.044921 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:28.049211 (+ 4290us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.18.195:47213: BlockingRecv error: recv got EOF from 127.8.18.195:47213 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
Aug 10 11:31:28 dist-test-slave-srgv krb5kdc[16743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.18.194@KRBTEST.COM for oryx/127.8.18.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 11:31:28 dist-test-slave-srgv krb5kdc[16743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.18.194@KRBTEST.COM for oryx/127.8.18.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 11:31:28.563581 17800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:28.553623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:35349 (local address 127.8.18.194:56041)
0810 11:31:28.554307 (+ 684us) negotiation.cc:107] Waiting for socket to connect
0810 11:31:28.554331 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0810 11:31:28.554489 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:31:28.555290 (+ 801us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:31:28.555297 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:31:28.556073 (+ 776us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:31:28.556297 (+ 224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:28.556307 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:28.557654 (+ 1347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:28.557659 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:28.558637 (+ 978us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:28.558650 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:28.559379 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:28.559384 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:28.559527 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:28.560255 (+ 728us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:31:28.560275 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:31:28.563313 (+ 3038us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.18.254:35349: Server oryx/127.8.18.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":539,"thread_start_us":316,"threads_started":1}
W20260810 11:31:28.563968 17801 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:31:28.554287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.194:56041 (local address 127.8.18.254:35349)
0810 11:31:28.554894 (+ 607us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:28.554899 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:28.554915 (+ 16us) server_negotiation.cc:407] Connection header received
0810 11:31:28.554997 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:28.555002 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:28.555101 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:28.555230 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:31:28.556454 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:28.557354 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:28.558779 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:28.559220 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:28.559318 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:28.563595 (+ 4277us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.18.194:56041: BlockingRecv error: recv got EOF from 127.8.18.194:56041 (error 108)
Metrics: {"server-negotiator.queue_time_us":513,"thread_start_us":427,"threads_started":1}
Aug 10 11:31:29 dist-test-slave-srgv krb5kdc[16743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.18.193@KRBTEST.COM for oryx/127.8.18.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 11:31:29 dist-test-slave-srgv krb5kdc[16743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.18.193@KRBTEST.COM for oryx/127.8.18.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 11:31:29.024828 17802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:29.014461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:35349 (local address 127.8.18.193:52687)
0810 11:31:29.015101 (+ 640us) negotiation.cc:107] Waiting for socket to connect
0810 11:31:29.015119 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 11:31:29.015258 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:31:29.015799 (+ 541us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:31:29.015806 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:31:29.016339 (+ 533us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:31:29.016591 (+ 252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:29.016606 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:29.018455 (+ 1849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:29.018461 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:29.019507 (+ 1046us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:29.019523 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:29.020337 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:29.020342 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:29.020491 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:29.021201 (+ 710us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:31:29.021221 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:31:29.024571 (+ 3350us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.18.254:35349: Server oryx/127.8.18.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":513,"thread_start_us":380,"threads_started":1}
W20260810 11:31:29.025063 17801 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:31:29.014673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.193:52687 (local address 127.8.18.254:35349)
0810 11:31:29.014885 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:29.014890 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:29.015226 (+ 336us) server_negotiation.cc:407] Connection header received
0810 11:31:29.015487 (+ 261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:29.015493 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:29.015584 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:29.015718 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:31:29.016828 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:29.018269 (+ 1441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:29.019718 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:29.020171 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:29.020288 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:29.024834 (+ 4546us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.18.193:52687: BlockingRecv error: recv got EOF from 127.8.18.193:52687 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
Aug 10 11:31:29 dist-test-slave-srgv krb5kdc[16743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.18.195@KRBTEST.COM for oryx/127.8.18.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 11:31:29 dist-test-slave-srgv krb5kdc[16743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.18.195@KRBTEST.COM for oryx/127.8.18.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 11:31:29.060250 17803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:29.050419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:35349 (local address 127.8.18.195:38289)
0810 11:31:29.051044 (+ 625us) negotiation.cc:107] Waiting for socket to connect
0810 11:31:29.051062 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 11:31:29.051200 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:31:29.051627 (+ 427us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:31:29.051631 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:31:29.052057 (+ 426us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:31:29.052250 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:29.052261 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:29.053347 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:29.053351 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:29.054448 (+ 1097us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:29.054467 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:29.055439 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:29.055444 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:29.055665 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:29.056538 (+ 873us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:31:29.056563 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:31:29.059979 (+ 3416us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.18.254:35349: Server oryx/127.8.18.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":486,"thread_start_us":327,"threads_started":1}
W20260810 11:31:29.060457 17801 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:31:29.050574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.195:38289 (local address 127.8.18.254:35349)
0810 11:31:29.050824 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:29.050829 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:29.051166 (+ 337us) server_negotiation.cc:407] Connection header received
0810 11:31:29.051328 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:29.051333 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:29.051422 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:29.051527 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:31:29.052428 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:29.053193 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:29.054636 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:29.055243 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:29.055358 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:29.060207 (+ 4849us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.18.195:38289: BlockingRecv error: recv got EOF from 127.8.18.195:38289 (error 108)
Metrics: {"server-negotiator.queue_time_us":111}
Aug 10 11:31:29 dist-test-slave-srgv krb5kdc[16743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.18.194@KRBTEST.COM for oryx/127.8.18.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 11:31:29 dist-test-slave-srgv krb5kdc[16743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.18.194@KRBTEST.COM for oryx/127.8.18.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 11:31:29.574554 17804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:29.564740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:35349 (local address 127.8.18.194:60605)
0810 11:31:29.565353 (+ 613us) negotiation.cc:107] Waiting for socket to connect
0810 11:31:29.565371 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 11:31:29.565567 (+ 196us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:31:29.566072 (+ 505us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:31:29.566075 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:31:29.566492 (+ 417us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:31:29.566653 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:29.566662 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:29.567721 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:29.567725 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:29.568722 (+ 997us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:29.568737 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:29.569770 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:29.569776 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:29.569936 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:29.570757 (+ 821us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:31:29.570779 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:31:29.574214 (+ 3435us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.18.254:35349: Server oryx/127.8.18.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":500,"thread_start_us":362,"threads_started":1}
W20260810 11:31:29.574800 17805 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:31:29.564849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.194:60605 (local address 127.8.18.254:35349)
0810 11:31:29.565711 (+ 862us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:29.565716 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:29.565735 (+ 19us) server_negotiation.cc:407] Connection header received
0810 11:31:29.565788 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:29.565792 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:29.565870 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:29.565967 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:31:29.566807 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:29.567552 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:29.568953 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:29.569550 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:29.569677 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:29.574500 (+ 4823us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.18.194:60605: BlockingRecv error: recv got EOF from 127.8.18.194:60605 (error 108)
Metrics: {"server-negotiator.queue_time_us":759,"thread_start_us":647,"threads_started":1}
Aug 10 11:31:30 dist-test-slave-srgv krb5kdc[16743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.18.193@KRBTEST.COM for oryx/127.8.18.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 11:31:30 dist-test-slave-srgv krb5kdc[16743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.18.193@KRBTEST.COM for oryx/127.8.18.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 11:31:30.038014 17806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:30.025947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:35349 (local address 127.8.18.193:43133)
0810 11:31:30.026815 (+ 868us) negotiation.cc:107] Waiting for socket to connect
0810 11:31:30.026832 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 11:31:30.026958 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:31:30.027420 (+ 462us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:31:30.027424 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:31:30.028034 (+ 610us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:31:30.028220 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:30.028230 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:30.030061 (+ 1831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:30.030067 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:30.031156 (+ 1089us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:30.031171 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:30.032388 (+ 1217us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:30.032393 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:30.032531 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:30.033360 (+ 829us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:31:30.033384 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:31:30.037674 (+ 4290us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.18.254:35349: Server oryx/127.8.18.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":742,"thread_start_us":500,"threads_started":1}
W20260810 11:31:30.038337 17805 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:31:30.026079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.193:43133 (local address 127.8.18.254:35349)
0810 11:31:30.026345 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:30.026355 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:30.026944 (+ 589us) server_negotiation.cc:407] Connection header received
0810 11:31:30.027090 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:30.027098 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:30.027212 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:30.027358 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:31:30.028394 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:30.029862 (+ 1468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:30.031322 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:30.032190 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:30.032370 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:30.038016 (+ 5646us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.18.193:43133: BlockingRecv error: recv got EOF from 127.8.18.193:43133 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
Aug 10 11:31:30 dist-test-slave-srgv krb5kdc[16743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.18.195@KRBTEST.COM for oryx/127.8.18.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 11:31:30 dist-test-slave-srgv krb5kdc[16743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.18.195@KRBTEST.COM for oryx/127.8.18.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 11:31:30.071959 17807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:30.061460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:35349 (local address 127.8.18.195:57077)
0810 11:31:30.062098 (+ 638us) negotiation.cc:107] Waiting for socket to connect
0810 11:31:30.062120 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 11:31:30.062286 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:31:30.062763 (+ 477us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:31:30.062766 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:31:30.063156 (+ 390us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:31:30.063326 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:30.063335 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:30.065155 (+ 1820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:30.065160 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:30.066223 (+ 1063us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:30.066239 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:30.067526 (+ 1287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:30.067531 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:30.067676 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:30.068458 (+ 782us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:31:30.068479 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:31:30.071705 (+ 3226us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.18.254:35349: Server oryx/127.8.18.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":502,"thread_start_us":407,"threads_started":1}
W20260810 11:31:30.072862 17805 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:31:30.061598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.195:57077 (local address 127.8.18.254:35349)
0810 11:31:30.061870 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:30.061879 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:30.062230 (+ 351us) server_negotiation.cc:407] Connection header received
0810 11:31:30.062468 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:30.062477 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:30.062588 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:30.062713 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:31:30.063494 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:30.064941 (+ 1447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:30.066398 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:30.067280 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:30.067465 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:30.072437 (+ 4972us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.18.195:57077: BlockingRecv error: recv got EOF from 127.8.18.195:57077 (error 108)
Metrics: {"server-negotiator.queue_time_us":87}
Aug 10 11:31:30 dist-test-slave-srgv krb5kdc[16743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.18.194@KRBTEST.COM for oryx/127.8.18.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 11:31:30 dist-test-slave-srgv krb5kdc[16743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.18.194@KRBTEST.COM for oryx/127.8.18.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 11:31:30.585011 17808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:30.575651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:35349 (local address 127.8.18.194:58253)
0810 11:31:30.576283 (+ 632us) negotiation.cc:107] Waiting for socket to connect
0810 11:31:30.576300 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 11:31:30.576438 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:31:30.576920 (+ 482us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:31:30.576924 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:31:30.577375 (+ 451us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:31:30.577621 (+ 246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:30.577631 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:30.578724 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:30.578729 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:30.579656 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:30.579668 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:30.580491 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:30.580497 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:30.580661 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:30.581325 (+ 664us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:31:30.581344 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:31:30.584721 (+ 3377us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.18.254:35349: Server oryx/127.8.18.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":506,"thread_start_us":292,"threads_started":1}
W20260810 11:31:30.585541 17809 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:31:30.575894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.194:58253 (local address 127.8.18.254:35349)
0810 11:31:30.576553 (+ 659us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:30.576559 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:30.576579 (+ 20us) server_negotiation.cc:407] Connection header received
0810 11:31:30.576680 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:30.576687 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:30.576780 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:30.576870 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:31:30.577770 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:30.578569 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:30.579809 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:30.580308 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:30.580413 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:30.585051 (+ 4638us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.18.194:58253: BlockingRecv error: recv got EOF from 127.8.18.194:58253 (error 108)
Metrics: {"server-negotiator.queue_time_us":471,"thread_start_us":360,"threads_started":1}
Aug 10 11:31:31 dist-test-slave-srgv krb5kdc[16743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.18.193@KRBTEST.COM for oryx/127.8.18.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 11:31:31 dist-test-slave-srgv krb5kdc[16743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.18.193@KRBTEST.COM for oryx/127.8.18.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 11:31:31.049350 17810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:31.039435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:35349 (local address 127.8.18.193:57285)
0810 11:31:31.040012 (+ 577us) negotiation.cc:107] Waiting for socket to connect
0810 11:31:31.040029 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 11:31:31.040149 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:31:31.040495 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:31:31.040498 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:31:31.041150 (+ 652us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:31:31.041334 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:31.041344 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:31.042551 (+ 1207us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:31.042555 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:31.043524 (+ 969us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:31.043537 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:31.044464 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:31.044469 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:31.044596 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:31.045340 (+ 744us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:31:31.045359 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:31:31.049054 (+ 3695us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.18.254:35349: Server oryx/127.8.18.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":428,"thread_start_us":307,"threads_started":1}
W20260810 11:31:31.049693 17809 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:31:31.039563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.193:57285 (local address 127.8.18.254:35349)
0810 11:31:31.039876 (+ 313us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:31.039882 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:31.040109 (+ 227us) server_negotiation.cc:407] Connection header received
0810 11:31:31.040266 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:31.040270 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:31.040353 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:31.040445 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:31:31.041601 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:31.042387 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:31.043700 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:31.044257 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:31.044376 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:31.049378 (+ 5002us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.18.193:57285: BlockingRecv error: recv got EOF from 127.8.18.193:57285 (error 108)
Metrics: {"server-negotiator.queue_time_us":91}
Aug 10 11:31:31 dist-test-slave-srgv krb5kdc[16743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.18.195@KRBTEST.COM for oryx/127.8.18.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 11:31:31 dist-test-slave-srgv krb5kdc[16743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.18.195@KRBTEST.COM for oryx/127.8.18.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 11:31:31.083504 17811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:31.073282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:35349 (local address 127.8.18.195:38251)
0810 11:31:31.074072 (+ 790us) negotiation.cc:107] Waiting for socket to connect
0810 11:31:31.074091 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 11:31:31.074276 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:31:31.074743 (+ 467us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:31:31.074750 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:31:31.075398 (+ 648us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:31:31.075649 (+ 251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:31.075663 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:31.077060 (+ 1397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:31.077064 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:31.078037 (+ 973us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:31.078052 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:31.078929 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:31.078934 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:31.079129 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:31.079791 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:31:31.079810 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:31:31.083207 (+ 3397us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.18.254:35349: Server oryx/127.8.18.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":653,"thread_start_us":338,"threads_started":1}
W20260810 11:31:31.083807 17809 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:31:31.073644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.195:38251 (local address 127.8.18.254:35349)
0810 11:31:31.073879 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:31.073887 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:31.074206 (+ 319us) server_negotiation.cc:407] Connection header received
0810 11:31:31.074415 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:31.074421 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:31.074502 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:31.074627 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:31:31.075795 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:31.076842 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:31.078229 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:31.078744 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:31.078901 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:31.083528 (+ 4627us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.18.195:38251: BlockingRecv error: recv got EOF from 127.8.18.195:38251 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
Aug 10 11:31:31 dist-test-slave-srgv krb5kdc[16743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.18.194@KRBTEST.COM for oryx/127.8.18.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 11:31:31 dist-test-slave-srgv krb5kdc[16743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.18.194@KRBTEST.COM for oryx/127.8.18.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 11:31:31.595772 17812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:31.586335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:35349 (local address 127.8.18.194:57661)
0810 11:31:31.586877 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0810 11:31:31.586902 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0810 11:31:31.587036 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:31:31.587521 (+ 485us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:31:31.587526 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:31:31.588038 (+ 512us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:31:31.588255 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:31.588265 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:31.589513 (+ 1248us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:31.589521 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:31.590670 (+ 1149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:31.590684 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:31.591482 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:31.591489 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:31.591664 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:31.592350 (+ 686us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:31:31.592369 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:31:31.595422 (+ 3053us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.18.254:35349: Server oryx/127.8.18.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":358,"thread_start_us":226,"threads_started":1}
W20260810 11:31:31.595986 17813 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:31:31.586461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.194:57661 (local address 127.8.18.254:35349)
0810 11:31:31.587179 (+ 718us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:31.587183 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:31.587202 (+ 19us) server_negotiation.cc:407] Connection header received
0810 11:31:31.587259 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:31.587263 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:31.587340 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:31.587439 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:31:31.588430 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:31.589333 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:31.590829 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:31.591290 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:31.591439 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:31.595698 (+ 4259us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.18.194:57661: BlockingRecv error: recv got EOF from 127.8.18.194:57661 (error 108)
Metrics: {"server-negotiator.queue_time_us":613,"thread_start_us":498,"threads_started":1}
Aug 10 11:31:32 dist-test-slave-srgv krb5kdc[16743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.18.193@KRBTEST.COM for oryx/127.8.18.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 11:31:32 dist-test-slave-srgv krb5kdc[16743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.18.193@KRBTEST.COM for oryx/127.8.18.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 11:31:32.060464 17814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:32.050685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:35349 (local address 127.8.18.193:51115)
0810 11:31:32.051321 (+ 636us) negotiation.cc:107] Waiting for socket to connect
0810 11:31:32.051339 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 11:31:32.051527 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:31:32.051927 (+ 400us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:31:32.051932 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:31:32.052457 (+ 525us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:31:32.052703 (+ 246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:32.052721 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:32.054088 (+ 1367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:32.054093 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:32.055130 (+ 1037us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:32.055145 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:32.055874 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:32.055880 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:32.056075 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:32.056818 (+ 743us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:31:32.056836 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:31:32.060070 (+ 3234us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.18.254:35349: Server oryx/127.8.18.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":526,"thread_start_us":338,"threads_started":1}
W20260810 11:31:32.060614 17813 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:31:32.050870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.193:51115 (local address 127.8.18.254:35349)
0810 11:31:32.051149 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:32.051156 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:32.051451 (+ 295us) server_negotiation.cc:407] Connection header received
0810 11:31:32.051670 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:32.051678 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:32.051760 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:32.051881 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:31:32.052886 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:32.053891 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:32.055296 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:32.055697 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:32.055847 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:32.060354 (+ 4507us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.18.193:51115: BlockingRecv error: recv got EOF from 127.8.18.193:51115 (error 108)
Metrics: {"server-negotiator.queue_time_us":101}
Aug 10 11:31:32 dist-test-slave-srgv krb5kdc[16743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.18.195@KRBTEST.COM for oryx/127.8.18.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 11:31:32 dist-test-slave-srgv krb5kdc[16743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.18.195@KRBTEST.COM for oryx/127.8.18.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 11:31:32.093976 17815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:32.084700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:35349 (local address 127.8.18.195:42531)
0810 11:31:32.085268 (+ 568us) negotiation.cc:107] Waiting for socket to connect
0810 11:31:32.085285 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 11:31:32.085480 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:31:32.085888 (+ 408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:31:32.085892 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:31:32.086283 (+ 391us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:31:32.086442 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:32.086450 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:32.087506 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:32.087511 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:32.088445 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:32.088458 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:32.089250 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:32.089255 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:32.089374 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:32.090104 (+ 730us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:31:32.090121 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:31:32.093642 (+ 3521us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.18.254:35349: Server oryx/127.8.18.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":420,"thread_start_us":262,"threads_started":1}
W20260810 11:31:32.094228 17813 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:31:32.084985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.195:42531 (local address 127.8.18.254:35349)
0810 11:31:32.085283 (+ 298us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:32.085288 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:32.085370 (+ 82us) server_negotiation.cc:407] Connection header received
0810 11:31:32.085620 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:32.085624 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:32.085698 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:32.085799 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:31:32.086584 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:32.087365 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:32.088595 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:32.089042 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:32.089138 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:32.093893 (+ 4755us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.18.195:42531: BlockingRecv error: recv got EOF from 127.8.18.195:42531 (error 108)
Metrics: {"server-negotiator.queue_time_us":146}
Aug 10 11:31:32 dist-test-slave-srgv krb5kdc[16743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.18.194@KRBTEST.COM for oryx/127.8.18.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 11:31:32 dist-test-slave-srgv krb5kdc[16743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.18.194@KRBTEST.COM for oryx/127.8.18.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 11:31:32.606791 17816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:32.596835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:35349 (local address 127.8.18.194:47039)
0810 11:31:32.597535 (+ 700us) negotiation.cc:107] Waiting for socket to connect
0810 11:31:32.597556 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 11:31:32.597672 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:31:32.598120 (+ 448us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:31:32.598125 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:31:32.598703 (+ 578us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:31:32.598908 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:32.598917 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:32.600068 (+ 1151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:32.600073 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:32.601174 (+ 1101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:32.601190 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:32.602248 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:32.602253 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:32.602412 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:32.603151 (+ 739us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:31:32.603171 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:31:32.606446 (+ 3275us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.18.254:35349: Server oryx/127.8.18.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":524,"thread_start_us":281,"threads_started":1}
W20260810 11:31:32.607297 17817 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:31:32.597023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.194:47039 (local address 127.8.18.254:35349)
0810 11:31:32.597778 (+ 755us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:32.597783 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:32.597800 (+ 17us) server_negotiation.cc:407] Connection header received
0810 11:31:32.597852 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:32.597856 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:32.597924 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:32.598022 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:31:32.599078 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:32.599907 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:32.601364 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:32.602061 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:32.602177 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:32.606896 (+ 4719us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.18.194:47039: BlockingRecv error: recv got EOF from 127.8.18.194:47039 (error 108)
Metrics: {"server-negotiator.queue_time_us":658,"thread_start_us":489,"threads_started":1}
Aug 10 11:31:33 dist-test-slave-srgv krb5kdc[16743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.18.193@KRBTEST.COM for oryx/127.8.18.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 11:31:33 dist-test-slave-srgv krb5kdc[16743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.18.193@KRBTEST.COM for oryx/127.8.18.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 11:31:33.074941 17818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:33.061847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:35349 (local address 127.8.18.193:36761)
0810 11:31:33.062495 (+ 648us) negotiation.cc:107] Waiting for socket to connect
0810 11:31:33.062513 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 11:31:33.062644 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:31:33.062974 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:31:33.062978 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:31:33.063437 (+ 459us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:31:33.063602 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:33.063611 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:33.064721 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:33.064727 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:33.065825 (+ 1098us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:33.065842 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:33.066635 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:33.066640 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:33.066778 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:33.070790 (+ 4012us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:31:33.070815 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:31:33.074646 (+ 3831us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.18.254:35349: Server oryx/127.8.18.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":538,"thread_start_us":415,"threads_started":1}
W20260810 11:31:33.075312 17817 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:31:33.061983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.193:36761 (local address 127.8.18.254:35349)
0810 11:31:33.062249 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:33.062259 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:33.062611 (+ 352us) server_negotiation.cc:407] Connection header received
0810 11:31:33.062759 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:33.062763 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:33.062830 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:33.062917 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:31:33.063759 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:33.064535 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:33.066017 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:33.066456 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:33.066564 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:33.074958 (+ 8394us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.18.193:36761: BlockingRecv error: recv got EOF from 127.8.18.193:36761 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Aug 10 11:31:33 dist-test-slave-srgv krb5kdc[16743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.18.195@KRBTEST.COM for oryx/127.8.18.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 11:31:33 dist-test-slave-srgv krb5kdc[16743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.18.195@KRBTEST.COM for oryx/127.8.18.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 11:31:33.105093 17819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:33.095080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:35349 (local address 127.8.18.195:35727)
0810 11:31:33.095743 (+ 663us) negotiation.cc:107] Waiting for socket to connect
0810 11:31:33.095763 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 11:31:33.095895 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:31:33.096364 (+ 469us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:31:33.096368 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:31:33.096796 (+ 428us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:31:33.096966 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:33.096974 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:33.098317 (+ 1343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:33.098322 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:33.099769 (+ 1447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:33.099822 (+ 53us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:33.100816 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:33.100823 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:33.101046 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:33.101849 (+ 803us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:31:33.101867 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:31:33.104818 (+ 2951us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.18.254:35349: Server oryx/127.8.18.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":537,"thread_start_us":379,"threads_started":1}
W20260810 11:31:33.105497 17817 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:31:33.095337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.195:35727 (local address 127.8.18.254:35349)
0810 11:31:33.095549 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:33.095554 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:33.095867 (+ 313us) server_negotiation.cc:407] Connection header received
0810 11:31:33.096069 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:33.096074 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:33.096202 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:33.096311 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:31:33.097231 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:33.098171 (+ 940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:33.100054 (+ 1883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:33.100625 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:33.100771 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:33.105114 (+ 4343us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.18.195:35727: BlockingRecv error: recv got EOF from 127.8.18.195:35727 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Aug 10 11:31:33 dist-test-slave-srgv krb5kdc[16743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.18.194@KRBTEST.COM for oryx/127.8.18.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 11:31:33 dist-test-slave-srgv krb5kdc[16743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.18.194@KRBTEST.COM for oryx/127.8.18.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 11:31:33.616940 17820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:33.608013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:35349 (local address 127.8.18.194:47789)
0810 11:31:33.608674 (+ 661us) negotiation.cc:107] Waiting for socket to connect
0810 11:31:33.608698 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0810 11:31:33.608828 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:31:33.609196 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:31:33.609200 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:31:33.609796 (+ 596us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:31:33.609979 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:33.609990 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:33.611047 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:33.611050 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:33.611998 (+ 948us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:33.612011 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:33.612812 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:33.612816 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:33.612936 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:33.613670 (+ 734us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:31:33.613688 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:31:33.616685 (+ 2997us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.18.254:35349: Server oryx/127.8.18.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":498,"thread_start_us":322,"threads_started":1}
W20260810 11:31:33.617319 17821 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:31:33.608057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.194:47789 (local address 127.8.18.254:35349)
0810 11:31:33.608854 (+ 797us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:33.608860 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:33.608881 (+ 21us) server_negotiation.cc:407] Connection header received
0810 11:31:33.608946 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:33.608951 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:33.609024 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:33.609114 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:31:33.610120 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:33.610915 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:33.612172 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:33.612659 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:33.612768 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:33.617001 (+ 4233us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.18.194:47789: BlockingRecv error: recv got EOF from 127.8.18.194:47789 (error 108)
Metrics: {"server-negotiator.queue_time_us":682,"thread_start_us":522,"threads_started":1}
Aug 10 11:31:34 dist-test-slave-srgv krb5kdc[16743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.18.193@KRBTEST.COM for oryx/127.8.18.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 11:31:34 dist-test-slave-srgv krb5kdc[16743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.18.193@KRBTEST.COM for oryx/127.8.18.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 11:31:34.086033 17822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:34.076306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:35349 (local address 127.8.18.193:52089)
0810 11:31:34.076926 (+ 620us) negotiation.cc:107] Waiting for socket to connect
0810 11:31:34.076944 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 11:31:34.077131 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:31:34.077738 (+ 607us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:31:34.077742 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:31:34.078187 (+ 445us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:31:34.078442 (+ 255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:34.078452 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:34.079598 (+ 1146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:34.079622 (+ 24us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:34.080550 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:34.080565 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:34.081394 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:34.081400 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:34.081641 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:34.082364 (+ 723us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:31:34.082382 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:31:34.085647 (+ 3265us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.18.254:35349: Server oryx/127.8.18.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":517,"thread_start_us":359,"threads_started":1}
W20260810 11:31:34.086202 17821 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:31:34.076473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.193:52089 (local address 127.8.18.254:35349)
0810 11:31:34.076822 (+ 349us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:34.076830 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:34.077057 (+ 227us) server_negotiation.cc:407] Connection header received
0810 11:31:34.077275 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:34.077282 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:34.077358 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:34.077495 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:31:34.078636 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:34.079428 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:34.080768 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:34.081202 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:34.081361 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:34.085931 (+ 4570us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.18.193:52089: BlockingRecv error: recv got EOF from 127.8.18.193:52089 (error 108)
Metrics: {"server-negotiator.queue_time_us":187}
Aug 10 11:31:34 dist-test-slave-srgv krb5kdc[16743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.18.195@KRBTEST.COM for oryx/127.8.18.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 11:31:34 dist-test-slave-srgv krb5kdc[16743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.18.195@KRBTEST.COM for oryx/127.8.18.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 11:31:34.115748 17823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:34.106265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:35349 (local address 127.8.18.195:46459)
0810 11:31:34.106893 (+ 628us) negotiation.cc:107] Waiting for socket to connect
0810 11:31:34.106918 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0810 11:31:34.107036 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:31:34.107388 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:31:34.107391 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:31:34.107843 (+ 452us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:31:34.107995 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:34.108005 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:34.109062 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:34.109066 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:34.110125 (+ 1059us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:34.110138 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:34.110917 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:34.110922 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:34.111096 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:34.111824 (+ 728us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:31:34.111842 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:31:34.115406 (+ 3564us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.18.254:35349: Server oryx/127.8.18.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":513,"thread_start_us":392,"threads_started":1}
W20260810 11:31:34.115960 17821 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:31:34.106355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.195:46459 (local address 127.8.18.254:35349)
0810 11:31:34.106544 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:34.106549 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:34.106993 (+ 444us) server_negotiation.cc:407] Connection header received
0810 11:31:34.107143 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:34.107148 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:34.107235 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:34.107336 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:31:34.108138 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:34.108907 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:34.110280 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:34.110713 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:34.110829 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:34.115665 (+ 4836us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.18.195:46459: BlockingRecv error: recv got EOF from 127.8.18.195:46459 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 10 11:31:34 dist-test-slave-srgv krb5kdc[16743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.18.194@KRBTEST.COM for oryx/127.8.18.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 11:31:34 dist-test-slave-srgv krb5kdc[16743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.18.194@KRBTEST.COM for oryx/127.8.18.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 11:31:34.627502 17824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:34.618059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:35349 (local address 127.8.18.194:32907)
0810 11:31:34.618665 (+ 606us) negotiation.cc:107] Waiting for socket to connect
0810 11:31:34.618693 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0810 11:31:34.618806 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:31:34.619151 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:31:34.619154 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:31:34.619622 (+ 468us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:31:34.619819 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:34.619830 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:34.620954 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:34.620958 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:34.622006 (+ 1048us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:34.622047 (+ 41us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:34.622839 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:34.622844 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:34.622983 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:34.623623 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:31:34.623643 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:31:34.627148 (+ 3505us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.18.254:35349: Server oryx/127.8.18.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":420,"thread_start_us":255,"threads_started":1}
W20260810 11:31:34.627880 17825 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:31:34.618249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.194:32907 (local address 127.8.18.254:35349)
0810 11:31:34.618765 (+ 516us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:34.618770 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:34.618790 (+ 20us) server_negotiation.cc:407] Connection header received
0810 11:31:34.618922 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:34.618926 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:34.619000 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:34.619097 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:31:34.619980 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:34.620808 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:34.622212 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:34.622675 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:34.622784 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:34.627521 (+ 4737us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.18.194:32907: BlockingRecv error: recv got EOF from 127.8.18.194:32907 (error 108)
Metrics: {"server-negotiator.queue_time_us":404,"thread_start_us":297,"threads_started":1}
Aug 10 11:31:35 dist-test-slave-srgv krb5kdc[16743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.18.193@KRBTEST.COM for oryx/127.8.18.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 11:31:35 dist-test-slave-srgv krb5kdc[16743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.18.193@KRBTEST.COM for oryx/127.8.18.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 11:31:35.097380 17826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:35.087384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:35349 (local address 127.8.18.193:51057)
0810 11:31:35.088018 (+ 634us) negotiation.cc:107] Waiting for socket to connect
0810 11:31:35.088035 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 11:31:35.088218 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:31:35.088630 (+ 412us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:31:35.088636 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:31:35.089092 (+ 456us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:31:35.089265 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:35.089274 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:35.090441 (+ 1167us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:35.090447 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:35.091435 (+ 988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:35.091449 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:35.092262 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:35.092269 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:35.092466 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:35.093236 (+ 770us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:31:35.093256 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:31:35.097106 (+ 3850us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.18.254:35349: Server oryx/127.8.18.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":533,"thread_start_us":348,"threads_started":1}
W20260810 11:31:35.097781 17825 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:31:35.087567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.193:51057 (local address 127.8.18.254:35349)
0810 11:31:35.087862 (+ 295us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:35.087870 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:35.088144 (+ 274us) server_negotiation.cc:407] Connection header received
0810 11:31:35.088381 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:35.088386 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:35.088455 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:35.088580 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:31:35.089461 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:35.090271 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:35.091615 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:35.092073 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:35.092226 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:35.097400 (+ 5174us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.18.193:51057: BlockingRecv error: recv got EOF from 127.8.18.193:51057 (error 108)
Metrics: {"server-negotiator.queue_time_us":87}
Aug 10 11:31:35 dist-test-slave-srgv krb5kdc[16743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.18.195@KRBTEST.COM for oryx/127.8.18.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 11:31:35 dist-test-slave-srgv krb5kdc[16743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.18.195@KRBTEST.COM for oryx/127.8.18.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 11:31:35.127240 17827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:35.117062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:35349 (local address 127.8.18.195:51139)
0810 11:31:35.117667 (+ 605us) negotiation.cc:107] Waiting for socket to connect
0810 11:31:35.117686 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 11:31:35.117845 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:31:35.118190 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:31:35.118193 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:31:35.118666 (+ 473us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:31:35.118888 (+ 222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:35.118900 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:35.120290 (+ 1390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:35.120295 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:35.121503 (+ 1208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:35.121522 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:35.122259 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:35.122265 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:35.122398 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:35.123075 (+ 677us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:31:35.123093 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:31:35.126831 (+ 3738us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.18.254:35349: Server oryx/127.8.18.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":503,"thread_start_us":392,"threads_started":1}
W20260810 11:31:35.127446 17825 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:31:35.117260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.195:51139 (local address 127.8.18.254:35349)
0810 11:31:35.117567 (+ 307us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:35.117573 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:35.117811 (+ 238us) server_negotiation.cc:407] Connection header received
0810 11:31:35.117955 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:35.117960 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:35.118045 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:35.118142 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:31:35.119093 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:35.120088 (+ 995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:35.121655 (+ 1567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:35.122086 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:35.122199 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:35.127136 (+ 4937us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.18.195:51139: BlockingRecv error: recv got EOF from 127.8.18.195:51139 (error 108)
Metrics: {"server-negotiator.queue_time_us":113}
Aug 10 11:31:35 dist-test-slave-srgv krb5kdc[16743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.18.194@KRBTEST.COM for oryx/127.8.18.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 11:31:35 dist-test-slave-srgv krb5kdc[16743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.18.194@KRBTEST.COM for oryx/127.8.18.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 11:31:35.638708 17828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:35.628771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:35349 (local address 127.8.18.194:46485)
0810 11:31:35.629478 (+ 707us) negotiation.cc:107] Waiting for socket to connect
0810 11:31:35.629504 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0810 11:31:35.629662 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:31:35.630105 (+ 443us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:31:35.630110 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:31:35.630568 (+ 458us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:31:35.630734 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:35.630745 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:35.631893 (+ 1148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:35.631901 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:35.632854 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:35.632868 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:35.633916 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:35.633922 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:35.634142 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:35.634898 (+ 756us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:31:35.634929 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:31:35.638360 (+ 3431us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.18.254:35349: Server oryx/127.8.18.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":527,"thread_start_us":275,"threads_started":1}
W20260810 11:31:35.638972 17829 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:31:35.628915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.194:46485 (local address 127.8.18.254:35349)
0810 11:31:35.629615 (+ 700us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:35.629621 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:35.629643 (+ 22us) server_negotiation.cc:407] Connection header received
0810 11:31:35.629841 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:35.629845 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:35.629932 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:35.630059 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:31:35.630926 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:35.631709 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:35.633049 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:35.633733 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:35.633887 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:35.638635 (+ 4748us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.18.194:46485: BlockingRecv error: recv got EOF from 127.8.18.194:46485 (error 108)
Metrics: {"server-negotiator.queue_time_us":560,"thread_start_us":425,"threads_started":1}
Aug 10 11:31:36 dist-test-slave-srgv krb5kdc[16743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.18.193@KRBTEST.COM for oryx/127.8.18.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 11:31:36 dist-test-slave-srgv krb5kdc[16743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.18.193@KRBTEST.COM for oryx/127.8.18.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 11:31:36.109838 17830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:36.098838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:35349 (local address 127.8.18.193:60913)
0810 11:31:36.099596 (+ 758us) negotiation.cc:107] Waiting for socket to connect
0810 11:31:36.099623 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0810 11:31:36.099832 (+ 209us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:31:36.100304 (+ 472us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:31:36.100309 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:31:36.100874 (+ 565us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:31:36.101074 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:36.101086 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:36.102514 (+ 1428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:36.102519 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:36.103679 (+ 1160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:36.103696 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:36.104824 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:36.104830 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:36.105196 (+ 366us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:36.105943 (+ 747us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:31:36.105963 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:31:36.109389 (+ 3426us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.18.254:35349: Server oryx/127.8.18.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":592,"thread_start_us":376,"threads_started":1}
W20260810 11:31:36.110020 17829 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:31:36.098986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.193:60913 (local address 127.8.18.254:35349)
0810 11:31:36.099283 (+ 297us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:36.099291 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:36.099751 (+ 460us) server_negotiation.cc:407] Connection header received
0810 11:31:36.099988 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:36.099994 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:36.100078 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:36.100206 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:31:36.101275 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:36.102316 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:36.103852 (+ 1536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:36.104565 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:36.104694 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:36.109721 (+ 5027us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.18.193:60913: BlockingRecv error: recv got EOF from 127.8.18.193:60913 (error 108)
Metrics: {"server-negotiator.queue_time_us":104}
Aug 10 11:31:36 dist-test-slave-srgv krb5kdc[16743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.18.195@KRBTEST.COM for oryx/127.8.18.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 11:31:36 dist-test-slave-srgv krb5kdc[16743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.18.195@KRBTEST.COM for oryx/127.8.18.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 11:31:36.138403 17831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:36.128689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:35349 (local address 127.8.18.195:35417)
0810 11:31:36.129387 (+ 698us) negotiation.cc:107] Waiting for socket to connect
0810 11:31:36.129458 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0810 11:31:36.129645 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:31:36.130102 (+ 457us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:31:36.130105 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:31:36.130722 (+ 617us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:31:36.130952 (+ 230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:36.130968 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:36.132136 (+ 1168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:36.132140 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:36.133089 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:36.133103 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:36.133861 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:36.133865 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:36.134006 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:36.134683 (+ 677us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:31:36.134703 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:31:36.138126 (+ 3423us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.18.254:35349: Server oryx/127.8.18.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":576,"thread_start_us":385,"threads_started":1}
W20260810 11:31:36.138792 17829 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:31:36.128920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.195:35417 (local address 127.8.18.254:35349)
0810 11:31:36.129143 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:36.129151 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:36.129582 (+ 431us) server_negotiation.cc:407] Connection header received
0810 11:31:36.129809 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:36.129815 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:36.129922 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:36.130040 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:31:36.131127 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:36.131984 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:36.133265 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:36.133710 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:36.133826 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:36.138423 (+ 4597us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.18.195:35417: BlockingRecv error: recv got EOF from 127.8.18.195:35417 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 10 11:31:36 dist-test-slave-srgv krb5kdc[16743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.18.194@KRBTEST.COM for oryx/127.8.18.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 11:31:36 dist-test-slave-srgv krb5kdc[16743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.18.194@KRBTEST.COM for oryx/127.8.18.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 11:31:36.654204 17832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:36.639764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:35349 (local address 127.8.18.194:38765)
0810 11:31:36.640357 (+ 593us) negotiation.cc:107] Waiting for socket to connect
0810 11:31:36.640384 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0810 11:31:36.640499 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:31:36.640849 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:31:36.640853 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:31:36.641271 (+ 418us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:31:36.641516 (+ 245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:36.641528 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:36.642683 (+ 1155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:36.642687 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:36.643658 (+ 971us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:36.643672 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:36.644571 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:36.644576 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:36.644715 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:36.650131 (+ 5416us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:31:36.650170 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:31:36.653930 (+ 3760us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.18.254:35349: Server oryx/127.8.18.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":441,"thread_start_us":246,"threads_started":1}
W20260810 11:31:36.654624 17833 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:31:36.639977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.194:38765 (local address 127.8.18.254:35349)
0810 11:31:36.640461 (+ 484us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:36.640466 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:36.640485 (+ 19us) server_negotiation.cc:407] Connection header received
0810 11:31:36.640615 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:36.640620 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:36.640686 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:36.640784 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:31:36.641675 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:36.642505 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:36.643853 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:36.644361 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:36.644487 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:36.654318 (+ 9831us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.18.194:38765: BlockingRecv error: recv got EOF from 127.8.18.194:38765 (error 108)
Metrics: {"server-negotiator.queue_time_us":379,"thread_start_us":276,"threads_started":1}
Aug 10 11:31:37 dist-test-slave-srgv krb5kdc[16743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.18.193@KRBTEST.COM for oryx/127.8.18.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 11:31:37 dist-test-slave-srgv krb5kdc[16743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.18.193@KRBTEST.COM for oryx/127.8.18.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 11:31:37.120697 17834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:37.111071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:35349 (local address 127.8.18.193:37063)
0810 11:31:37.111605 (+ 534us) negotiation.cc:107] Waiting for socket to connect
0810 11:31:37.111622 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 11:31:37.111798 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:31:37.112178 (+ 380us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:31:37.112184 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:31:37.112622 (+ 438us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:31:37.112817 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:37.112827 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:37.114126 (+ 1299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:37.114138 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:37.115083 (+ 945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:37.115095 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:37.115985 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:37.116019 (+ 34us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:37.116211 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:37.117043 (+ 832us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:31:37.117063 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:31:37.120411 (+ 3348us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.18.254:35349: Server oryx/127.8.18.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":423,"thread_start_us":289,"threads_started":1}
W20260810 11:31:37.120923 17833 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:31:37.111259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.193:37063 (local address 127.8.18.254:35349)
0810 11:31:37.111498 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:37.111505 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:37.111717 (+ 212us) server_negotiation.cc:407] Connection header received
0810 11:31:37.111954 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:37.111958 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:37.112025 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:37.112140 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:31:37.112986 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:37.113956 (+ 970us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:37.115251 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:37.115787 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:37.115925 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:37.120690 (+ 4765us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.18.193:37063: BlockingRecv error: recv got EOF from 127.8.18.193:37063 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
Aug 10 11:31:37 dist-test-slave-srgv krb5kdc[16743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.18.195@KRBTEST.COM for oryx/127.8.18.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 11:31:37 dist-test-slave-srgv krb5kdc[16743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.18.195@KRBTEST.COM for oryx/127.8.18.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 11:31:37.150348 17835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:37.139531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:35349 (local address 127.8.18.195:52823)
0810 11:31:37.140189 (+ 658us) negotiation.cc:107] Waiting for socket to connect
0810 11:31:37.140209 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 11:31:37.140401 (+ 192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:31:37.140833 (+ 432us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:31:37.140838 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:31:37.141332 (+ 494us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:31:37.141590 (+ 258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:37.141600 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:37.143217 (+ 1617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:37.143226 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:37.144459 (+ 1233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:37.144475 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:37.145477 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:37.145486 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:37.145644 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:37.146389 (+ 745us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:31:37.146411 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:31:37.150043 (+ 3632us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.18.254:35349: Server oryx/127.8.18.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":515,"thread_start_us":262,"threads_started":1}
W20260810 11:31:37.150712 17833 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:31:37.139734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.195:52823 (local address 127.8.18.254:35349)
0810 11:31:37.140123 (+ 389us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:37.140131 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:37.140326 (+ 195us) server_negotiation.cc:407] Connection header received
0810 11:31:37.140552 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:37.140560 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:37.140647 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:37.140791 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:31:37.141799 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:37.142984 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:37.144691 (+ 1707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:37.145194 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:37.145312 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:37.150355 (+ 5043us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.18.195:52823: BlockingRecv error: recv got EOF from 127.8.18.195:52823 (error 108)
Metrics: {"server-negotiator.queue_time_us":217}
Aug 10 11:31:37 dist-test-slave-srgv krb5kdc[16743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.18.194@KRBTEST.COM for oryx/127.8.18.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 11:31:37 dist-test-slave-srgv krb5kdc[16743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.18.194@KRBTEST.COM for oryx/127.8.18.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 11:31:37.664829 17836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:37.655476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:35349 (local address 127.8.18.194:40275)
0810 11:31:37.656096 (+ 620us) negotiation.cc:107] Waiting for socket to connect
0810 11:31:37.656126 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0810 11:31:37.656309 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:31:37.656758 (+ 449us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:31:37.656764 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:31:37.657307 (+ 543us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:31:37.657599 (+ 292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:37.657613 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:37.658648 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:37.658654 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:37.659577 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:37.659590 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:37.660357 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:37.660363 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:37.660555 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:37.661276 (+ 721us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:31:37.661296 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:31:37.664463 (+ 3167us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.18.254:35349: Server oryx/127.8.18.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":443,"thread_start_us":284,"threads_started":1}
W20260810 11:31:37.665053 17837 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:31:37.655618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.194:40275 (local address 127.8.18.254:35349)
0810 11:31:37.656378 (+ 760us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:37.656382 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:37.656398 (+ 16us) server_negotiation.cc:407] Connection header received
0810 11:31:37.656454 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:37.656458 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:37.656537 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:37.656627 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:31:37.657753 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:37.658489 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:37.659746 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:37.660186 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:37.660328 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:37.664780 (+ 4452us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.18.194:40275: BlockingRecv error: recv got EOF from 127.8.18.194:40275 (error 108)
Metrics: {"server-negotiator.queue_time_us":629,"thread_start_us":493,"threads_started":1}
Aug 10 11:31:38 dist-test-slave-srgv krb5kdc[16743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.18.193@KRBTEST.COM for oryx/127.8.18.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 11:31:38 dist-test-slave-srgv krb5kdc[16743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.18.193@KRBTEST.COM for oryx/127.8.18.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 11:31:38.134848 17838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:38.121955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:35349 (local address 127.8.18.193:38005)
0810 11:31:38.122520 (+ 565us) negotiation.cc:107] Waiting for socket to connect
0810 11:31:38.122539 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 11:31:38.122656 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:31:38.122968 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:31:38.122972 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:31:38.123425 (+ 453us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:31:38.123629 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:38.123639 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:38.124667 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:38.124671 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:38.125700 (+ 1029us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:38.125714 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:38.126476 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:38.126480 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:38.126596 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:38.130833 (+ 4237us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:31:38.130862 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:31:38.134420 (+ 3558us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.18.254:35349: Server oryx/127.8.18.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":437,"thread_start_us":319,"threads_started":1}
W20260810 11:31:38.134909 17837 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:31:38.122063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.193:38005 (local address 127.8.18.254:35349)
0810 11:31:38.122284 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:38.122288 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:38.122630 (+ 342us) server_negotiation.cc:407] Connection header received
0810 11:31:38.122763 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:38.122767 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:38.122832 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:38.122921 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:31:38.123781 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:38.124523 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:38.125887 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:38.126331 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:38.126446 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:38.134667 (+ 8221us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.18.193:38005: BlockingRecv error: recv got EOF from 127.8.18.193:38005 (error 108)
Metrics: {"server-negotiator.queue_time_us":105}
Aug 10 11:31:38 dist-test-slave-srgv krb5kdc[16743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.18.195@KRBTEST.COM for oryx/127.8.18.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 11:31:38 dist-test-slave-srgv krb5kdc[16743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.18.195@KRBTEST.COM for oryx/127.8.18.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 11:31:38.160773 17839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:38.151474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:35349 (local address 127.8.18.195:47239)
0810 11:31:38.152177 (+ 703us) negotiation.cc:107] Waiting for socket to connect
0810 11:31:38.152193 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 11:31:38.152374 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:31:38.152765 (+ 391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:31:38.152770 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:31:38.153208 (+ 438us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:31:38.153397 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:38.153407 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:38.154550 (+ 1143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:38.154556 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:38.155478 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:38.155490 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:38.156329 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:38.156335 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:38.156528 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:38.157354 (+ 826us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:31:38.157373 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:31:38.160373 (+ 3000us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.18.254:35349: Server oryx/127.8.18.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":544,"thread_start_us":397,"threads_started":1}
W20260810 11:31:38.160887 17837 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:31:38.151679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.195:47239 (local address 127.8.18.254:35349)
0810 11:31:38.152097 (+ 418us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:38.152104 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:38.152295 (+ 191us) server_negotiation.cc:407] Connection header received
0810 11:31:38.152517 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:38.152523 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:38.152614 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:38.152730 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:31:38.153622 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:38.154377 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:38.155655 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:38.156136 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:38.156312 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:38.160620 (+ 4308us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.18.195:47239: BlockingRecv error: recv got EOF from 127.8.18.195:47239 (error 108)
Metrics: {"server-negotiator.queue_time_us":179}
Aug 10 11:31:38 dist-test-slave-srgv krb5kdc[16743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.18.194@KRBTEST.COM for oryx/127.8.18.254@KRBTEST.COM, Server not found in Kerberos database
Aug 10 11:31:38 dist-test-slave-srgv krb5kdc[16743](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.18.194@KRBTEST.COM for oryx/127.8.18.254@KRBTEST.COM, Server not found in Kerberos database
I20260810 11:31:38.675325 17840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:38.666246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:35349 (local address 127.8.18.194:42589)
0810 11:31:38.667068 (+ 822us) negotiation.cc:107] Waiting for socket to connect
0810 11:31:38.667087 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 11:31:38.667191 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:31:38.667515 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:31:38.667518 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:31:38.667974 (+ 456us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:31:38.668176 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:38.668187 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:38.669305 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:38.669309 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:38.670356 (+ 1047us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:38.670369 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:38.671057 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:38.671061 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:38.671175 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:38.671820 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:31:38.671839 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:31:38.674978 (+ 3139us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.18.254:35349: Server oryx/127.8.18.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":678,"thread_start_us":523,"threads_started":1}
W20260810 11:31:38.675702 17841 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:31:38.666381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.194:42589 (local address 127.8.18.254:35349)
0810 11:31:38.667104 (+ 723us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:38.667108 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:38.667137 (+ 29us) server_negotiation.cc:407] Connection header received
0810 11:31:38.667302 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:38.667306 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:38.667379 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:38.667470 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:31:38.668338 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:38.669153 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:38.670497 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:38.670921 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:38.671022 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:38.675298 (+ 4276us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.18.194:42589: BlockingRecv error: recv got EOF from 127.8.18.194:42589 (error 108)
Metrics: {"server-negotiator.queue_time_us":576,"thread_start_us":450,"threads_started":1}
I20260810 11:31:38.982425 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 17354
I20260810 11:31:38.996474 17843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:38.983805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50468 (local address 127.8.18.193:36787)
0810 11:31:38.984597 (+ 792us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:38.984607 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:38.984841 (+ 234us) server_negotiation.cc:407] Connection header received
0810 11:31:38.984964 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:38.984972 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:38.985255 (+ 283us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:38.985442 (+ 187us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:31:38.987124 (+ 1682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:38.988171 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:38.989633 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:38.990237 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:38.990354 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:38.993261 (+ 2907us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:31:38.993271 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:31:38.993287 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:31:38.993324 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:31:38.995944 (+ 2620us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.8.18.193@KRBTEST.COM not found in keytab (ticket kvno 2)
0810 11:31:38.996239 (+ 295us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:50468: Request ticket server kudu/127.8.18.193@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":597,"thread_start_us":353,"threads_started":1}
W20260810 11:31:38.996622 17843 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:50468: Request ticket server kudu/127.8.18.193@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260810 11:31:38.996819 8267 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 17354: Not authorized: Client connection negotiation failed: client connection to 127.8.18.193:36787: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.8.18.193@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260810 11:31:38.997069 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 17354
I20260810 11:31:39.016319 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 17509
I20260810 11:31:39.030022 17845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:39.017639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47164 (local address 127.8.18.194:44667)
0810 11:31:39.018366 (+ 727us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:39.018373 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:39.018406 (+ 33us) server_negotiation.cc:407] Connection header received
0810 11:31:39.018472 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:39.018478 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:39.018707 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:39.018895 (+ 188us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:31:39.020542 (+ 1647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:39.021785 (+ 1243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:39.023020 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:39.023597 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:39.023701 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:39.026565 (+ 2864us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:31:39.026576 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:31:39.026591 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:31:39.026638 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:31:39.029395 (+ 2757us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.8.18.194@KRBTEST.COM not found in keytab (ticket kvno 2)
0810 11:31:39.029756 (+ 361us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:47164: Request ticket server kudu/127.8.18.194@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":553,"thread_start_us":399,"threads_started":1}
W20260810 11:31:39.030171 17845 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:47164: Request ticket server kudu/127.8.18.194@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260810 11:31:39.030239 8267 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 17509: Not authorized: Client connection negotiation failed: client connection to 127.8.18.194:44667: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.8.18.194@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260810 11:31:39.030470 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 17509
I20260810 11:31:39.048971 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 17645
I20260810 11:31:39.062021 17846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:39.050112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54186 (local address 127.8.18.195:45471)
0810 11:31:39.050748 (+ 636us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:39.050755 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:39.050784 (+ 29us) server_negotiation.cc:407] Connection header received
0810 11:31:39.050845 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:39.050851 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:39.051067 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:39.051220 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:31:39.052763 (+ 1543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:39.053935 (+ 1172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:39.055197 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:39.055833 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:39.055934 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:39.058812 (+ 2878us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:31:39.058821 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:31:39.058836 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:31:39.058875 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:31:39.061365 (+ 2490us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.8.18.195@KRBTEST.COM not found in keytab (ticket kvno 2)
0810 11:31:39.061739 (+ 374us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:54186: Request ticket server kudu/127.8.18.195@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":465,"thread_start_us":343,"threads_started":1}
W20260810 11:31:39.062140 17846 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:54186: Request ticket server kudu/127.8.18.195@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260810 11:31:39.062399 8267 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 17645: Not authorized: Client connection negotiation failed: client connection to 127.8.18.195:45471: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.8.18.195@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260810 11:31:39.062538 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 17645
I20260810 11:31:39.080705 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 17283
I20260810 11:31:39.082250 17347 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:31:39.209046 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 17283
2026-08-10T11:31:39Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (21956 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-10T11:31:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T11:31:39Z Disabled control of system clock
I20260810 11:31:39.302105 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786361401922362-8267-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786361401922362-8267-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786361401922362-8267-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.18.254:43389
--webserver_interface=127.8.18.254
--webserver_port=0
--builtin_ntp_servers=127.8.18.212:37009
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.18.254:43389
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260810 11:31:39.674471 17852 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:31:39.674826 17852 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:31:39.674914 17852 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:31:39.682765 17852 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 11:31:39.682904 17852 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:31:39.682942 17852 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:31:39.683004 17852 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 11:31:39.683051 17852 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 11:31:39.693074 17852 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.18.212:37009
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786361401922362-8267-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786361401922362-8267-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.18.254:43389
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.18.254:43389
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.18.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786361401922362-8267-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:31:39.694911 17852 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:31:39.696933 17852 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:31:39.711081 17857 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:31:39.710984 17860 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:31:39.710984 17858 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:31:39.712242 17852 server_base.cc:1034] running on GCE node
I20260810 11:31:39.713208 17852 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:31:39.714902 17852 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:31:39.716162 17852 hybrid_clock.cc:648] HybridClock initialized: now 1786361499716123 us; error 73 us; skew 500 ppm
I20260810 11:31:39.716596 17852 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 11:31:39.719337 17852 webserver.cc:460] Webserver started at http://127.8.18.254:44449/ using document root <none> and password file <none>
I20260810 11:31:39.720149 17852 fs_manager.cc:359] Metadata directory not provided
I20260810 11:31:39.720305 17852 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:31:39.720666 17852 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:31:39.723989 17852 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786361401922362-8267-0/minicluster-data/master-0/data/instance:
uuid: "f836d5340fe644fc8119f4b9f39375d7"
format_stamp: "Formatted at 2026-08-10 11:31:39 on dist-test-slave-srgv"
server_key: "58a3bac22d4ffd65a2eb656d63d369bb"
server_key_iv: "352a6f1c0f6622d6994184206b495247"
server_key_version: "encryptionkey@0"
I20260810 11:31:39.724928 17852 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786361401922362-8267-0/minicluster-data/master-0/wal/instance:
uuid: "f836d5340fe644fc8119f4b9f39375d7"
format_stamp: "Formatted at 2026-08-10 11:31:39 on dist-test-slave-srgv"
server_key: "58a3bac22d4ffd65a2eb656d63d369bb"
server_key_iv: "352a6f1c0f6622d6994184206b495247"
server_key_version: "encryptionkey@0"
I20260810 11:31:39.730803 17852 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260810 11:31:39.734892 17866 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:31:39.736611 17852 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260810 11:31:39.736869 17852 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786361401922362-8267-0/minicluster-data/master-0/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786361401922362-8267-0/minicluster-data/master-0/wal
uuid: "f836d5340fe644fc8119f4b9f39375d7"
format_stamp: "Formatted at 2026-08-10 11:31:39 on dist-test-slave-srgv"
server_key: "58a3bac22d4ffd65a2eb656d63d369bb"
server_key_iv: "352a6f1c0f6622d6994184206b495247"
server_key_version: "encryptionkey@0"
I20260810 11:31:39.737160 17852 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786361401922362-8267-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786361401922362-8267-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786361401922362-8267-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:31:39.754278 17852 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:31:39.755313 17852 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:31:39.755668 17852 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:31:39.776902 17852 rpc_server.cc:307] RPC server started. Bound to: 127.8.18.254:43389
I20260810 11:31:39.776923 17920 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.18.254:43389 every 8 connection(s)
I20260810 11:31:39.778528 17852 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
I20260810 11:31:39.781590 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 17852
I20260810 11:31:39.781838 8267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786361401922362-8267-0/minicluster-data/master-0/wal/instance
I20260810 11:31:39.782539 8267 external_mini_cluster.cc:1428] Setting key 728990e80765d74f88c14f4749f94391
I20260810 11:31:39.783200 17921 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:31:39.800648 17921 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f836d5340fe644fc8119f4b9f39375d7: Bootstrap starting.
I20260810 11:31:39.801270 17924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:39.784942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41344 (local address 127.8.18.254:43389)
0810 11:31:39.786311 (+ 1369us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:39.786338 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:39.786415 (+ 77us) server_negotiation.cc:407] Connection header received
0810 11:31:39.788925 (+ 2510us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:39.788970 (+ 45us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:39.789861 (+ 891us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:39.790462 (+ 601us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:31:39.791203 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:39.793372 (+ 2169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:39.795050 (+ 1678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:39.796203 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:39.796425 (+ 222us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:39.797871 (+ 1446us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:31:39.797882 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:31:39.797906 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:31:39.798749 (+ 843us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:31:39.798866 (+ 117us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:31:39.798895 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:31:39.799033 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0810 11:31:39.799274 (+ 241us) server_negotiation.cc:299] Negotiation successful
0810 11:31:39.799720 (+ 446us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":747,"thread_start_us":477,"threads_started":1}
I20260810 11:31:39.804433 17921 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f836d5340fe644fc8119f4b9f39375d7: Neither blocks nor log segments found. Creating new log.
I20260810 11:31:39.805755 17921 log.cc:824] T 00000000000000000000000000000000 P f836d5340fe644fc8119f4b9f39375d7: Log is configured to *not* fsync() on all Append() calls
I20260810 11:31:39.809005 17921 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f836d5340fe644fc8119f4b9f39375d7: No bootstrap required, opened a new log
I20260810 11:31:39.817689 17921 raft_consensus.cc:348] T 00000000000000000000000000000000 P f836d5340fe644fc8119f4b9f39375d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f836d5340fe644fc8119f4b9f39375d7" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 43389 } }
I20260810 11:31:39.818181 17921 raft_consensus.cc:374] T 00000000000000000000000000000000 P f836d5340fe644fc8119f4b9f39375d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:31:39.818316 17921 raft_consensus.cc:729] T 00000000000000000000000000000000 P f836d5340fe644fc8119f4b9f39375d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f836d5340fe644fc8119f4b9f39375d7, State: Initialized, Role: FOLLOWER
I20260810 11:31:39.819018 17921 consensus_queue.cc:260] T 00000000000000000000000000000000 P f836d5340fe644fc8119f4b9f39375d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f836d5340fe644fc8119f4b9f39375d7" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 43389 } }
I20260810 11:31:39.819310 17921 raft_consensus.cc:388] T 00000000000000000000000000000000 P f836d5340fe644fc8119f4b9f39375d7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 11:31:39.819482 17921 raft_consensus.cc:482] T 00000000000000000000000000000000 P f836d5340fe644fc8119f4b9f39375d7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 11:31:39.819696 17921 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f836d5340fe644fc8119f4b9f39375d7 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:31:39.822268 17921 raft_consensus.cc:504] T 00000000000000000000000000000000 P f836d5340fe644fc8119f4b9f39375d7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f836d5340fe644fc8119f4b9f39375d7" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 43389 } }
I20260810 11:31:39.822877 17921 leader_election.cc:302] T 00000000000000000000000000000000 P f836d5340fe644fc8119f4b9f39375d7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f836d5340fe644fc8119f4b9f39375d7; no voters:
I20260810 11:31:39.823608 17921 leader_election.cc:288] T 00000000000000000000000000000000 P f836d5340fe644fc8119f4b9f39375d7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 11:31:39.823838 17926 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f836d5340fe644fc8119f4b9f39375d7 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:31:39.825011 17926 raft_consensus.cc:686] T 00000000000000000000000000000000 P f836d5340fe644fc8119f4b9f39375d7 [term 1 LEADER]: Becoming Leader. State: Replica: f836d5340fe644fc8119f4b9f39375d7, State: Running, Role: LEADER
I20260810 11:31:39.826066 17926 consensus_queue.cc:237] T 00000000000000000000000000000000 P f836d5340fe644fc8119f4b9f39375d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f836d5340fe644fc8119f4b9f39375d7" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 43389 } }
I20260810 11:31:39.826260 17921 sys_catalog.cc:564] T 00000000000000000000000000000000 P f836d5340fe644fc8119f4b9f39375d7 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 11:31:39.829631 17928 sys_catalog.cc:455] T 00000000000000000000000000000000 P f836d5340fe644fc8119f4b9f39375d7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f836d5340fe644fc8119f4b9f39375d7. Latest consensus state: current_term: 1 leader_uuid: "f836d5340fe644fc8119f4b9f39375d7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f836d5340fe644fc8119f4b9f39375d7" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 43389 } } }
I20260810 11:31:39.829968 17927 sys_catalog.cc:455] T 00000000000000000000000000000000 P f836d5340fe644fc8119f4b9f39375d7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f836d5340fe644fc8119f4b9f39375d7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f836d5340fe644fc8119f4b9f39375d7" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 43389 } } }
I20260810 11:31:39.830539 17927 sys_catalog.cc:458] T 00000000000000000000000000000000 P f836d5340fe644fc8119f4b9f39375d7 [sys.catalog]: This master's current role is: LEADER
I20260810 11:31:39.830689 17928 sys_catalog.cc:458] T 00000000000000000000000000000000 P f836d5340fe644fc8119f4b9f39375d7 [sys.catalog]: This master's current role is: LEADER
I20260810 11:31:39.834199 17935 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 11:31:39.839221 17935 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 11:31:39.848501 17935 catalog_manager.cc:1347] Generated new cluster ID: d68b30f254d942bbbc28b172a2de2932
I20260810 11:31:39.848654 17935 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 11:31:39.873857 17935 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 11:31:39.875346 17935 catalog_manager.cc:1504] Loading token signing keys...
I20260810 11:31:39.882233 17935 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f836d5340fe644fc8119f4b9f39375d7: Generated new TSK 0
I20260810 11:31:39.883225 17935 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 11:31:39.893142 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786361401922362-8267-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786361401922362-8267-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786361401922362-8267-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786361401922362-8267-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.18.193:0
--local_ip_for_outbound_sockets=127.8.18.193
--webserver_interface=127.8.18.193
--webserver_port=0
--tserver_master_addrs=127.8.18.254:43389
--builtin_ntp_servers=127.8.18.212:37009
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260810 11:31:40.271279 17947 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:31:40.271613 17947 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:31:40.271706 17947 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:31:40.279335 17947 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:31:40.279469 17947 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:31:40.279582 17947 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.18.193
I20260810 11:31:40.289232 17947 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.18.212:37009
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786361401922362-8267-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786361401922362-8267-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.18.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786361401922362-8267-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.18.193
--webserver_port=0
--tserver_master_addrs=127.8.18.254:43389
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.18.193
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786361401922362-8267-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:31:40.290639 17947 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:31:40.292596 17947 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:31:40.307608 17954 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:31:40.307689 17956 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:31:40.308251 17953 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:31:40.308521 17947 server_base.cc:1034] running on GCE node
I20260810 11:31:40.309283 17947 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:31:40.310446 17947 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:31:40.311748 17947 hybrid_clock.cc:648] HybridClock initialized: now 1786361500311675 us; error 55 us; skew 500 ppm
I20260810 11:31:40.312150 17947 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 11:31:40.314930 17947 webserver.cc:460] Webserver started at http://127.8.18.193:40805/ using document root <none> and password file <none>
I20260810 11:31:40.315757 17947 fs_manager.cc:359] Metadata directory not provided
I20260810 11:31:40.315864 17947 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:31:40.316249 17947 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:31:40.319454 17947 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786361401922362-8267-0/minicluster-data/ts-0/data/instance:
uuid: "8279361ae0624d11ab62e418e69801ac"
format_stamp: "Formatted at 2026-08-10 11:31:40 on dist-test-slave-srgv"
server_key: "2d4333de3e56f60c8488bdee88bb2e6d"
server_key_iv: "de43695b26f62923b1035cf131c87991"
server_key_version: "encryptionkey@0"
I20260810 11:31:40.320417 17947 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786361401922362-8267-0/minicluster-data/ts-0/wal/instance:
uuid: "8279361ae0624d11ab62e418e69801ac"
format_stamp: "Formatted at 2026-08-10 11:31:40 on dist-test-slave-srgv"
server_key: "2d4333de3e56f60c8488bdee88bb2e6d"
server_key_iv: "de43695b26f62923b1035cf131c87991"
server_key_version: "encryptionkey@0"
I20260810 11:31:40.326424 17947 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.003s
I20260810 11:31:40.330715 17963 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:31:40.332446 17947 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.002s
I20260810 11:31:40.332674 17947 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786361401922362-8267-0/minicluster-data/ts-0/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786361401922362-8267-0/minicluster-data/ts-0/wal
uuid: "8279361ae0624d11ab62e418e69801ac"
format_stamp: "Formatted at 2026-08-10 11:31:40 on dist-test-slave-srgv"
server_key: "2d4333de3e56f60c8488bdee88bb2e6d"
server_key_iv: "de43695b26f62923b1035cf131c87991"
server_key_version: "encryptionkey@0"
I20260810 11:31:40.332998 17947 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786361401922362-8267-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786361401922362-8267-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786361401922362-8267-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:31:40.353101 17947 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:31:40.354198 17947 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:31:40.354586 17947 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:31:40.355747 17947 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:31:40.357520 17947 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:31:40.357663 17947 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:31:40.357817 17947 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:31:40.357913 17947 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:31:40.395151 17947 rpc_server.cc:307] RPC server started. Bound to: 127.8.18.193:34203
I20260810 11:31:40.395265 18081 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.18.193:34203 every 8 connection(s)
I20260810 11:31:40.396670 17947 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786361401922362-8267-0/minicluster-data/ts-0/data/info.pb
I20260810 11:31:40.403926 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 17947
I20260810 11:31:40.404253 8267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786361401922362-8267-0/minicluster-data/ts-0/wal/instance
I20260810 11:31:40.405341 8267 external_mini_cluster.cc:1428] Setting key 076919f4147cdc26aea297c4a2910447
I20260810 11:31:40.409255 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786361401922362-8267-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786361401922362-8267-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786361401922362-8267-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786361401922362-8267-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.18.194:0
--local_ip_for_outbound_sockets=127.8.18.194
--webserver_interface=127.8.18.194
--webserver_port=0
--tserver_master_addrs=127.8.18.254:43389
--builtin_ntp_servers=127.8.18.212:37009
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260810 11:31:40.413918 18084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:40.399720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.193:57283 (local address 127.8.18.254:43389)
0810 11:31:40.400214 (+ 494us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:40.400219 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:40.401871 (+ 1652us) server_negotiation.cc:407] Connection header received
0810 11:31:40.403839 (+ 1968us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:40.403844 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:40.403949 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:40.404096 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:31:40.406145 (+ 2049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:40.407128 (+ 983us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:40.410244 (+ 3116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:40.410692 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:40.410834 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:40.412951 (+ 2117us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:31:40.412956 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:31:40.412959 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:31:40.412982 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:31:40.413031 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:31:40.413044 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:31:40.413116 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0810 11:31:40.413515 (+ 399us) server_negotiation.cc:299] Negotiation successful
0810 11:31:40.413619 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":360,"thread_start_us":241,"threads_started":1}
I20260810 11:31:40.415645 18085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:40.400435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:43389 (local address 127.8.18.193:57283)
0810 11:31:40.401628 (+ 1193us) negotiation.cc:107] Waiting for socket to connect
0810 11:31:40.401670 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 11:31:40.403462 (+ 1792us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:31:40.404526 (+ 1064us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:31:40.404557 (+ 31us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:31:40.404725 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:31:40.405875 (+ 1150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:40.405910 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:40.407365 (+ 1455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:40.407370 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:40.409985 (+ 2615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:40.410023 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:40.411020 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:40.411031 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:40.411446 (+ 415us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:40.412441 (+ 995us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 11:31:40.412448 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:31:40.412515 (+ 67us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 11:31:40.412660 (+ 145us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:31:40.413159 (+ 499us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:31:40.413174 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:31:40.413187 (+ 13us) client_negotiation.cc:769] Sending connection context
0810 11:31:40.413439 (+ 252us) client_negotiation.cc:240] Negotiation successful
0810 11:31:40.413984 (+ 545us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":584,"thread_start_us":355,"threads_started":1}
I20260810 11:31:40.417577 18082 heartbeater.cc:347] Connected to a master server at 127.8.18.254:43389
I20260810 11:31:40.417941 18082 heartbeater.cc:464] Registering TS with master...
I20260810 11:31:40.418843 18082 heartbeater.cc:511] Master 127.8.18.254:43389 requested a full tablet report, sending...
I20260810 11:31:40.421334 17886 ts_manager.cc:194] Registered new tserver with Master: 8279361ae0624d11ab62e418e69801ac (127.8.18.193:34203)
I20260810 11:31:40.423048 17886 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.18.193:57283
W20260810 11:31:40.766883 18086 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:31:40.767314 18086 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:31:40.767488 18086 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:31:40.775518 18086 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:31:40.775731 18086 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:31:40.775916 18086 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.18.194
I20260810 11:31:40.786814 18086 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.18.212:37009
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786361401922362-8267-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786361401922362-8267-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.18.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786361401922362-8267-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.18.194
--webserver_port=0
--tserver_master_addrs=127.8.18.254:43389
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.18.194
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786361401922362-8267-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:31:40.788307 18086 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:31:40.790334 18086 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:31:40.805390 18092 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:31:40.805433 18094 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:31:40.805737 18091 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:31:40.806777 18086 server_base.cc:1034] running on GCE node
I20260810 11:31:40.807636 18086 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:31:40.808893 18086 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:31:40.810161 18086 hybrid_clock.cc:648] HybridClock initialized: now 1786361500810104 us; error 82 us; skew 500 ppm
I20260810 11:31:40.810559 18086 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 11:31:40.813333 18086 webserver.cc:460] Webserver started at http://127.8.18.194:42817/ using document root <none> and password file <none>
I20260810 11:31:40.814277 18086 fs_manager.cc:359] Metadata directory not provided
I20260810 11:31:40.814448 18086 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:31:40.814842 18086 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:31:40.818395 18086 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786361401922362-8267-0/minicluster-data/ts-1/data/instance:
uuid: "507cd963235b473f9c81ecb13334a702"
format_stamp: "Formatted at 2026-08-10 11:31:40 on dist-test-slave-srgv"
server_key: "3e8e1e8d90aaf7ec3ddfd1b29057eff8"
server_key_iv: "1837293e87260b54429e0578a721cca7"
server_key_version: "encryptionkey@0"
I20260810 11:31:40.819476 18086 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786361401922362-8267-0/minicluster-data/ts-1/wal/instance:
uuid: "507cd963235b473f9c81ecb13334a702"
format_stamp: "Formatted at 2026-08-10 11:31:40 on dist-test-slave-srgv"
server_key: "3e8e1e8d90aaf7ec3ddfd1b29057eff8"
server_key_iv: "1837293e87260b54429e0578a721cca7"
server_key_version: "encryptionkey@0"
I20260810 11:31:40.825850 18086 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.003s
I20260810 11:31:40.830668 18100 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:31:40.832429 18086 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260810 11:31:40.832685 18086 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786361401922362-8267-0/minicluster-data/ts-1/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786361401922362-8267-0/minicluster-data/ts-1/wal
uuid: "507cd963235b473f9c81ecb13334a702"
format_stamp: "Formatted at 2026-08-10 11:31:40 on dist-test-slave-srgv"
server_key: "3e8e1e8d90aaf7ec3ddfd1b29057eff8"
server_key_iv: "1837293e87260b54429e0578a721cca7"
server_key_version: "encryptionkey@0"
I20260810 11:31:40.832948 18086 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786361401922362-8267-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786361401922362-8267-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786361401922362-8267-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:31:40.855995 18086 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:31:40.857015 18086 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:31:40.857455 18086 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:31:40.858619 18086 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:31:40.860356 18086 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:31:40.860491 18086 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:31:40.860625 18086 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:31:40.860718 18086 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:31:40.897614 18086 rpc_server.cc:307] RPC server started. Bound to: 127.8.18.194:39059
I20260810 11:31:40.897710 18216 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.18.194:39059 every 8 connection(s)
I20260810 11:31:40.898952 18086 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786361401922362-8267-0/minicluster-data/ts-1/data/info.pb
I20260810 11:31:40.900815 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 18086
I20260810 11:31:40.901029 8267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786361401922362-8267-0/minicluster-data/ts-1/wal/instance
I20260810 11:31:40.901695 8267 external_mini_cluster.cc:1428] Setting key 14a434a7ba80ddc617f5fb98ba7dc5d2
I20260810 11:31:40.905037 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786361401922362-8267-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786361401922362-8267-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786361401922362-8267-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786361401922362-8267-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.18.195:0
--local_ip_for_outbound_sockets=127.8.18.195
--webserver_interface=127.8.18.195
--webserver_port=0
--tserver_master_addrs=127.8.18.254:43389
--builtin_ntp_servers=127.8.18.212:37009
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260810 11:31:40.913606 18084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:40.901849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.194:40929 (local address 127.8.18.254:43389)
0810 11:31:40.902044 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:40.902049 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:40.903778 (+ 1729us) server_negotiation.cc:407] Connection header received
0810 11:31:40.905957 (+ 2179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:40.905964 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:40.906081 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:40.906206 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:31:40.907514 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:40.908269 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:40.910116 (+ 1847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:40.910564 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:40.910724 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:40.912507 (+ 1783us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:31:40.912513 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:31:40.912516 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:31:40.912540 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:31:40.912586 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:31:40.912601 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:31:40.912727 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0810 11:31:40.913021 (+ 294us) server_negotiation.cc:299] Negotiation successful
0810 11:31:40.913161 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260810 11:31:40.915182 18219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:40.902519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:43389 (local address 127.8.18.194:40929)
0810 11:31:40.903576 (+ 1057us) negotiation.cc:107] Waiting for socket to connect
0810 11:31:40.903623 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0810 11:31:40.905624 (+ 2001us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:31:40.906480 (+ 856us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:31:40.906491 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:31:40.906628 (+ 137us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:31:40.907295 (+ 667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:40.907315 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:40.908469 (+ 1154us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:40.908474 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:40.909915 (+ 1441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:40.909930 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:40.910837 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:40.910845 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:40.911116 (+ 271us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:40.912022 (+ 906us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 11:31:40.912026 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:31:40.912086 (+ 60us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 11:31:40.912205 (+ 119us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:31:40.912730 (+ 525us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:31:40.912747 (+ 17us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:31:40.912758 (+ 11us) client_negotiation.cc:769] Sending connection context
0810 11:31:40.913013 (+ 255us) client_negotiation.cc:240] Negotiation successful
0810 11:31:40.913674 (+ 661us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":572,"thread_start_us":612,"threads_started":1}
I20260810 11:31:40.917315 18217 heartbeater.cc:347] Connected to a master server at 127.8.18.254:43389
I20260810 11:31:40.917776 18217 heartbeater.cc:464] Registering TS with master...
I20260810 11:31:40.918931 18217 heartbeater.cc:511] Master 127.8.18.254:43389 requested a full tablet report, sending...
I20260810 11:31:40.921121 17886 ts_manager.cc:194] Registered new tserver with Master: 507cd963235b473f9c81ecb13334a702 (127.8.18.194:39059)
I20260810 11:31:40.922106 17886 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.18.194:40929
W20260810 11:31:41.266570 18220 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:31:41.266892 18220 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:31:41.267102 18220 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:31:41.274757 18220 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:31:41.274895 18220 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:31:41.275099 18220 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.18.195
I20260810 11:31:41.284780 18220 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.18.212:37009
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786361401922362-8267-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786361401922362-8267-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.18.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786361401922362-8267-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.18.195
--webserver_port=0
--tserver_master_addrs=127.8.18.254:43389
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.18.195
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786361401922362-8267-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:31:41.286305 18220 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:31:41.288255 18220 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:31:41.307440 18226 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:31:41.307803 18225 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:31:41.307897 18228 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:31:41.308787 18220 server_base.cc:1034] running on GCE node
I20260810 11:31:41.309635 18220 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:31:41.310782 18220 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:31:41.312091 18220 hybrid_clock.cc:648] HybridClock initialized: now 1786361501312026 us; error 55 us; skew 500 ppm
I20260810 11:31:41.312493 18220 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 11:31:41.315356 18220 webserver.cc:460] Webserver started at http://127.8.18.195:40993/ using document root <none> and password file <none>
I20260810 11:31:41.316215 18220 fs_manager.cc:359] Metadata directory not provided
I20260810 11:31:41.316385 18220 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:31:41.316768 18220 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:31:41.320230 18220 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786361401922362-8267-0/minicluster-data/ts-2/data/instance:
uuid: "7107915d206a47e6ab270b08bcee8e1e"
format_stamp: "Formatted at 2026-08-10 11:31:41 on dist-test-slave-srgv"
server_key: "cc9ff748f2eba56d52ae94ef9ea06332"
server_key_iv: "15e8e165aceeabc253b9f59687376150"
server_key_version: "encryptionkey@0"
I20260810 11:31:41.321189 18220 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786361401922362-8267-0/minicluster-data/ts-2/wal/instance:
uuid: "7107915d206a47e6ab270b08bcee8e1e"
format_stamp: "Formatted at 2026-08-10 11:31:41 on dist-test-slave-srgv"
server_key: "cc9ff748f2eba56d52ae94ef9ea06332"
server_key_iv: "15e8e165aceeabc253b9f59687376150"
server_key_version: "encryptionkey@0"
I20260810 11:31:41.327466 18220 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.000s
I20260810 11:31:41.332115 18234 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:31:41.333984 18220 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260810 11:31:41.334249 18220 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786361401922362-8267-0/minicluster-data/ts-2/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786361401922362-8267-0/minicluster-data/ts-2/wal
uuid: "7107915d206a47e6ab270b08bcee8e1e"
format_stamp: "Formatted at 2026-08-10 11:31:41 on dist-test-slave-srgv"
server_key: "cc9ff748f2eba56d52ae94ef9ea06332"
server_key_iv: "15e8e165aceeabc253b9f59687376150"
server_key_version: "encryptionkey@0"
I20260810 11:31:41.334529 18220 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786361401922362-8267-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786361401922362-8267-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786361401922362-8267-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:31:41.367724 18220 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:31:41.368822 18220 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:31:41.369223 18220 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:31:41.370467 18220 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:31:41.372457 18220 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:31:41.372617 18220 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:31:41.372781 18220 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:31:41.372884 18220 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:31:41.411034 18220 rpc_server.cc:307] RPC server started. Bound to: 127.8.18.195:34577
I20260810 11:31:41.411156 18350 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.18.195:34577 every 8 connection(s)
I20260810 11:31:41.412496 18220 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786361401922362-8267-0/minicluster-data/ts-2/data/info.pb
I20260810 11:31:41.419703 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 18220
I20260810 11:31:41.419962 8267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786361401922362-8267-0/minicluster-data/ts-2/wal/instance
I20260810 11:31:41.420532 8267 external_mini_cluster.cc:1428] Setting key e6b5dd62d8c18f477884bec5b48a4918
I20260810 11:31:41.426654 18082 heartbeater.cc:503] Master 127.8.18.254:43389 was elected leader, sending a full tablet report...
I20260810 11:31:41.427433 18353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:41.415608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.195:51743 (local address 127.8.18.254:43389)
0810 11:31:41.416173 (+ 565us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:41.416180 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:41.417606 (+ 1426us) server_negotiation.cc:407] Connection header received
0810 11:31:41.419639 (+ 2033us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:41.419643 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:41.419719 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:41.419878 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:31:41.421476 (+ 1598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:41.422464 (+ 988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:41.424116 (+ 1652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:41.424527 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:41.424681 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:41.426288 (+ 1607us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:31:41.426301 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:31:41.426306 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:31:41.426349 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:31:41.426408 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:31:41.426425 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:31:41.426542 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0810 11:31:41.426854 (+ 312us) server_negotiation.cc:299] Negotiation successful
0810 11:31:41.426964 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":429,"thread_start_us":313,"threads_started":1}
I20260810 11:31:41.428797 18354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:41.416321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:43389 (local address 127.8.18.195:51743)
0810 11:31:41.417349 (+ 1028us) negotiation.cc:107] Waiting for socket to connect
0810 11:31:41.417390 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0810 11:31:41.419263 (+ 1873us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:31:41.420183 (+ 920us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:31:41.420197 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:31:41.420333 (+ 136us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:31:41.421233 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:41.421255 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:41.422639 (+ 1384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:41.422644 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:41.423923 (+ 1279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:41.423936 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:41.424795 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:41.424803 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:41.425060 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:41.425873 (+ 813us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 11:31:41.425876 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:31:41.425967 (+ 91us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 11:31:41.426072 (+ 105us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:31:41.426583 (+ 511us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:31:41.426599 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:31:41.426612 (+ 13us) client_negotiation.cc:769] Sending connection context
0810 11:31:41.426821 (+ 209us) client_negotiation.cc:240] Negotiation successful
0810 11:31:41.427294 (+ 473us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":543,"thread_start_us":328,"threads_started":1}
I20260810 11:31:41.430292 18351 heartbeater.cc:347] Connected to a master server at 127.8.18.254:43389
I20260810 11:31:41.430610 18351 heartbeater.cc:464] Registering TS with master...
I20260810 11:31:41.431502 18351 heartbeater.cc:511] Master 127.8.18.254:43389 requested a full tablet report, sending...
I20260810 11:31:41.433203 17886 ts_manager.cc:194] Registered new tserver with Master: 7107915d206a47e6ab270b08bcee8e1e (127.8.18.195:34577)
I20260810 11:31:41.434290 17886 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.18.195:51743
I20260810 11:31:41.436887 8267 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260810 11:31:41.444535 18353 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:31:41.442423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41360 (local address 127.8.18.254:43389)
0810 11:31:41.442659 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:41.442666 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:41.443255 (+ 589us) server_negotiation.cc:407] Connection header received
0810 11:31:41.443487 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:41.443493 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:41.443595 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:41.443718 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:31:41.444263 (+ 545us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41360: BlockingRecv error: recv got EOF from 127.0.0.1:41360 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
I20260810 11:31:41.445447 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 17947
I20260810 11:31:41.454452 18361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:41.446672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48138 (local address 127.8.18.193:34203)
0810 11:31:41.447355 (+ 683us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:41.447371 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:41.447400 (+ 29us) server_negotiation.cc:407] Connection header received
0810 11:31:41.447462 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:41.447468 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:41.447687 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:41.447852 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:31:41.448312 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:41.449337 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:41.450798 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:41.451424 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:41.451538 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:41.452782 (+ 1244us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:31:41.452791 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:31:41.452807 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:31:41.453501 (+ 694us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:31:41.453556 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:31:41.453579 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:31:41.453684 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0810 11:31:41.453856 (+ 172us) server_negotiation.cc:299] Negotiation successful
0810 11:31:41.453991 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":504,"thread_start_us":336,"threads_started":1}
I20260810 11:31:41.456123 18077 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:31:41.543358 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 17947
I20260810 11:31:41.563958 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 18086
I20260810 11:31:41.572477 18363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:41.565019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35618 (local address 127.8.18.194:39059)
0810 11:31:41.565703 (+ 684us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:41.565710 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:41.565741 (+ 31us) server_negotiation.cc:407] Connection header received
0810 11:31:41.565804 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:41.565811 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:41.566034 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:41.566167 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:31:41.566592 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:41.567596 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:41.568874 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:41.569621 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:41.569726 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:41.570877 (+ 1151us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:31:41.570885 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:31:41.570899 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:31:41.571552 (+ 653us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:31:41.571602 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:31:41.571623 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:31:41.571718 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0810 11:31:41.571919 (+ 201us) server_negotiation.cc:299] Negotiation successful
0810 11:31:41.572081 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":532,"thread_start_us":400,"threads_started":1}
I20260810 11:31:41.573972 18212 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:31:41.660367 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 18086
I20260810 11:31:41.678274 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 18220
I20260810 11:31:41.686748 18365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:41.679364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38996 (local address 127.8.18.195:34577)
0810 11:31:41.680003 (+ 639us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:41.680010 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:41.680041 (+ 31us) server_negotiation.cc:407] Connection header received
0810 11:31:41.680104 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:41.680112 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:41.680324 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:41.680459 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:31:41.680899 (+ 440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:41.682038 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:41.683245 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:41.683820 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:41.683918 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:41.685052 (+ 1134us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:31:41.685060 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:31:41.685073 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:31:41.685865 (+ 792us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:31:41.685914 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:31:41.685947 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:31:41.686040 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0810 11:31:41.686191 (+ 151us) server_negotiation.cc:299] Negotiation successful
0810 11:31:41.686347 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":499,"thread_start_us":343,"threads_started":1}
I20260810 11:31:41.688043 18346 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:31:41.772277 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 18220
I20260810 11:31:41.793562 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 17852
I20260810 11:31:41.794939 17916 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:31:41.903517 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 17852
2026-08-10T11:31:41Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (2689 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-10T11:31:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T11:31:41Z Disabled control of system clock
I20260810 11:31:41.993263 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786361401922362-8267-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786361401922362-8267-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786361401922362-8267-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.18.254:45303
--webserver_interface=127.8.18.254
--webserver_port=0
--builtin_ntp_servers=127.8.18.212:41147
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.18.254:45303
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260810 11:31:42.366998 18371 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:31:42.367326 18371 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:31:42.367419 18371 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:31:42.375368 18371 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 11:31:42.375512 18371 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:31:42.375554 18371 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:31:42.375607 18371 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 11:31:42.375645 18371 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 11:31:42.385763 18371 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.18.212:41147
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786361401922362-8267-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786361401922362-8267-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.18.254:45303
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.18.254:45303
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.18.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786361401922362-8267-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:31:42.387228 18371 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:31:42.389231 18371 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:31:42.403692 18376 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:31:42.403580 18379 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:31:42.403627 18377 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:31:42.404968 18371 server_base.cc:1034] running on GCE node
I20260810 11:31:42.405992 18371 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:31:42.407661 18371 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:31:42.408922 18371 hybrid_clock.cc:648] HybridClock initialized: now 1786361502408826 us; error 92 us; skew 500 ppm
I20260810 11:31:42.409395 18371 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 11:31:42.412253 18371 webserver.cc:460] Webserver started at http://127.8.18.254:44833/ using document root <none> and password file <none>
I20260810 11:31:42.413074 18371 fs_manager.cc:359] Metadata directory not provided
I20260810 11:31:42.413178 18371 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:31:42.413666 18371 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:31:42.417045 18371 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786361401922362-8267-0/minicluster-data/master-0/data/instance:
uuid: "29d848b26bcc4c9ea5159e577ededd1c"
format_stamp: "Formatted at 2026-08-10 11:31:42 on dist-test-slave-srgv"
server_key: "eaf898fdb6f9cce3df1803176b0a8edf"
server_key_iv: "0aa4f5c63d18c9cb5754b5bb63a99f8e"
server_key_version: "encryptionkey@0"
I20260810 11:31:42.418143 18371 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786361401922362-8267-0/minicluster-data/master-0/wal/instance:
uuid: "29d848b26bcc4c9ea5159e577ededd1c"
format_stamp: "Formatted at 2026-08-10 11:31:42 on dist-test-slave-srgv"
server_key: "eaf898fdb6f9cce3df1803176b0a8edf"
server_key_iv: "0aa4f5c63d18c9cb5754b5bb63a99f8e"
server_key_version: "encryptionkey@0"
I20260810 11:31:42.424494 18371 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260810 11:31:42.428574 18385 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:31:42.430289 18371 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260810 11:31:42.430510 18371 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786361401922362-8267-0/minicluster-data/master-0/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786361401922362-8267-0/minicluster-data/master-0/wal
uuid: "29d848b26bcc4c9ea5159e577ededd1c"
format_stamp: "Formatted at 2026-08-10 11:31:42 on dist-test-slave-srgv"
server_key: "eaf898fdb6f9cce3df1803176b0a8edf"
server_key_iv: "0aa4f5c63d18c9cb5754b5bb63a99f8e"
server_key_version: "encryptionkey@0"
I20260810 11:31:42.430773 18371 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786361401922362-8267-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786361401922362-8267-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786361401922362-8267-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:31:42.437533 18371 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:31:42.438555 18371 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:31:42.438944 18371 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:31:42.461634 18371 rpc_server.cc:307] RPC server started. Bound to: 127.8.18.254:45303
I20260810 11:31:42.461661 18438 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.18.254:45303 every 8 connection(s)
I20260810 11:31:42.463570 18371 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
I20260810 11:31:42.467836 18439 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:31:42.473363 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 18371
I20260810 11:31:42.473631 8267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786361401922362-8267-0/minicluster-data/master-0/wal/instance
I20260810 11:31:42.474386 8267 external_mini_cluster.cc:1428] Setting key c0d2b2d79cd3e6c9f532293d4120a4f5
I20260810 11:31:42.480769 18439 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 29d848b26bcc4c9ea5159e577ededd1c: Bootstrap starting.
I20260810 11:31:42.484522 18442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:42.476851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51608 (local address 127.8.18.254:45303)
0810 11:31:42.478138 (+ 1287us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:42.478149 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:42.478210 (+ 61us) server_negotiation.cc:407] Connection header received
0810 11:31:42.480109 (+ 1899us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:42.480139 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:42.480659 (+ 520us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:42.481158 (+ 499us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:31:42.482528 (+ 1370us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:31:42.482535 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:31:42.482550 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:31:42.482587 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:31:42.482658 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:31:42.482676 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:31:42.482960 (+ 284us) server_negotiation.cc:1035] Waiting for connection context
0810 11:31:42.483125 (+ 165us) server_negotiation.cc:299] Negotiation successful
0810 11:31:42.483447 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":733,"thread_start_us":484,"threads_started":1}
I20260810 11:31:42.484975 18439 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 29d848b26bcc4c9ea5159e577ededd1c: Neither blocks nor log segments found. Creating new log.
I20260810 11:31:42.486366 18439 log.cc:824] T 00000000000000000000000000000000 P 29d848b26bcc4c9ea5159e577ededd1c: Log is configured to *not* fsync() on all Append() calls
I20260810 11:31:42.489794 18439 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 29d848b26bcc4c9ea5159e577ededd1c: No bootstrap required, opened a new log
I20260810 11:31:42.497766 18439 raft_consensus.cc:348] T 00000000000000000000000000000000 P 29d848b26bcc4c9ea5159e577ededd1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29d848b26bcc4c9ea5159e577ededd1c" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 45303 } }
I20260810 11:31:42.498255 18439 raft_consensus.cc:374] T 00000000000000000000000000000000 P 29d848b26bcc4c9ea5159e577ededd1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:31:42.498378 18439 raft_consensus.cc:729] T 00000000000000000000000000000000 P 29d848b26bcc4c9ea5159e577ededd1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29d848b26bcc4c9ea5159e577ededd1c, State: Initialized, Role: FOLLOWER
I20260810 11:31:42.499106 18439 consensus_queue.cc:260] T 00000000000000000000000000000000 P 29d848b26bcc4c9ea5159e577ededd1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29d848b26bcc4c9ea5159e577ededd1c" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 45303 } }
I20260810 11:31:42.499396 18439 raft_consensus.cc:388] T 00000000000000000000000000000000 P 29d848b26bcc4c9ea5159e577ededd1c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 11:31:42.499558 18439 raft_consensus.cc:482] T 00000000000000000000000000000000 P 29d848b26bcc4c9ea5159e577ededd1c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 11:31:42.499753 18439 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 29d848b26bcc4c9ea5159e577ededd1c [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:31:42.502338 18439 raft_consensus.cc:504] T 00000000000000000000000000000000 P 29d848b26bcc4c9ea5159e577ededd1c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29d848b26bcc4c9ea5159e577ededd1c" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 45303 } }
I20260810 11:31:42.502991 18439 leader_election.cc:302] T 00000000000000000000000000000000 P 29d848b26bcc4c9ea5159e577ededd1c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 29d848b26bcc4c9ea5159e577ededd1c; no voters:
I20260810 11:31:42.503688 18439 leader_election.cc:288] T 00000000000000000000000000000000 P 29d848b26bcc4c9ea5159e577ededd1c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 11:31:42.503875 18444 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 29d848b26bcc4c9ea5159e577ededd1c [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:31:42.505259 18444 raft_consensus.cc:686] T 00000000000000000000000000000000 P 29d848b26bcc4c9ea5159e577ededd1c [term 1 LEADER]: Becoming Leader. State: Replica: 29d848b26bcc4c9ea5159e577ededd1c, State: Running, Role: LEADER
I20260810 11:31:42.506026 18444 consensus_queue.cc:237] T 00000000000000000000000000000000 P 29d848b26bcc4c9ea5159e577ededd1c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29d848b26bcc4c9ea5159e577ededd1c" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 45303 } }
I20260810 11:31:42.506357 18439 sys_catalog.cc:564] T 00000000000000000000000000000000 P 29d848b26bcc4c9ea5159e577ededd1c [sys.catalog]: configured and running, proceeding with master startup.
I20260810 11:31:42.509446 18445 sys_catalog.cc:455] T 00000000000000000000000000000000 P 29d848b26bcc4c9ea5159e577ededd1c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "29d848b26bcc4c9ea5159e577ededd1c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29d848b26bcc4c9ea5159e577ededd1c" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 45303 } } }
I20260810 11:31:42.510038 18445 sys_catalog.cc:458] T 00000000000000000000000000000000 P 29d848b26bcc4c9ea5159e577ededd1c [sys.catalog]: This master's current role is: LEADER
I20260810 11:31:42.509471 18447 sys_catalog.cc:455] T 00000000000000000000000000000000 P 29d848b26bcc4c9ea5159e577ededd1c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 29d848b26bcc4c9ea5159e577ededd1c. Latest consensus state: current_term: 1 leader_uuid: "29d848b26bcc4c9ea5159e577ededd1c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29d848b26bcc4c9ea5159e577ededd1c" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 45303 } } }
I20260810 11:31:42.510437 18447 sys_catalog.cc:458] T 00000000000000000000000000000000 P 29d848b26bcc4c9ea5159e577ededd1c [sys.catalog]: This master's current role is: LEADER
I20260810 11:31:42.519120 18454 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 11:31:42.525098 18454 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 11:31:42.534605 18454 catalog_manager.cc:1347] Generated new cluster ID: b26cbcec0bc34c57bad2d1d264bc9f5e
I20260810 11:31:42.534771 18454 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 11:31:42.557073 18454 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 11:31:42.557673 18454 catalog_manager.cc:1504] Loading token signing keys...
I20260810 11:31:42.569185 18454 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 29d848b26bcc4c9ea5159e577ededd1c: Generated new TSK 0
I20260810 11:31:42.570489 18454 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 11:31:42.591362 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786361401922362-8267-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786361401922362-8267-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786361401922362-8267-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786361401922362-8267-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.18.193:0
--local_ip_for_outbound_sockets=127.8.18.193
--webserver_interface=127.8.18.193
--webserver_port=0
--tserver_master_addrs=127.8.18.254:45303
--builtin_ntp_servers=127.8.18.212:41147
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260810 11:31:42.966969 18464 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:31:42.967288 18464 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:31:42.967387 18464 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:31:42.975100 18464 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:31:42.975234 18464 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:31:42.975350 18464 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.18.193
I20260810 11:31:42.985270 18464 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.18.212:41147
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786361401922362-8267-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786361401922362-8267-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.18.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786361401922362-8267-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.18.193
--webserver_port=0
--tserver_master_addrs=127.8.18.254:45303
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.18.193
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786361401922362-8267-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:31:42.986717 18464 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:31:42.988644 18464 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:31:43.004151 18472 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:31:43.004225 18469 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:31:43.004179 18470 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:31:43.005147 18464 server_base.cc:1034] running on GCE node
I20260810 11:31:43.005927 18464 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:31:43.007148 18464 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:31:43.008426 18464 hybrid_clock.cc:648] HybridClock initialized: now 1786361503008383 us; error 48 us; skew 500 ppm
I20260810 11:31:43.008816 18464 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 11:31:43.012004 18464 webserver.cc:460] Webserver started at http://127.8.18.193:45149/ using document root <none> and password file <none>
I20260810 11:31:43.012805 18464 fs_manager.cc:359] Metadata directory not provided
I20260810 11:31:43.012975 18464 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:31:43.013365 18464 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:31:43.016638 18464 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786361401922362-8267-0/minicluster-data/ts-0/data/instance:
uuid: "3d5cda76cb1f4241aacfe6c4a8d32672"
format_stamp: "Formatted at 2026-08-10 11:31:43 on dist-test-slave-srgv"
server_key: "0913e43f891da9fcd2e88d36b70e9546"
server_key_iv: "185d87b1b83f60aa8b638e4e2dbca80d"
server_key_version: "encryptionkey@0"
I20260810 11:31:43.017715 18464 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786361401922362-8267-0/minicluster-data/ts-0/wal/instance:
uuid: "3d5cda76cb1f4241aacfe6c4a8d32672"
format_stamp: "Formatted at 2026-08-10 11:31:43 on dist-test-slave-srgv"
server_key: "0913e43f891da9fcd2e88d36b70e9546"
server_key_iv: "185d87b1b83f60aa8b638e4e2dbca80d"
server_key_version: "encryptionkey@0"
I20260810 11:31:43.023836 18464 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260810 11:31:43.028110 18478 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:31:43.029848 18464 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 11:31:43.030072 18464 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786361401922362-8267-0/minicluster-data/ts-0/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786361401922362-8267-0/minicluster-data/ts-0/wal
uuid: "3d5cda76cb1f4241aacfe6c4a8d32672"
format_stamp: "Formatted at 2026-08-10 11:31:43 on dist-test-slave-srgv"
server_key: "0913e43f891da9fcd2e88d36b70e9546"
server_key_iv: "185d87b1b83f60aa8b638e4e2dbca80d"
server_key_version: "encryptionkey@0"
I20260810 11:31:43.030344 18464 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786361401922362-8267-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786361401922362-8267-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786361401922362-8267-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:31:43.037070 18464 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:31:43.038194 18464 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:31:43.038537 18464 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:31:43.039680 18464 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:31:43.041519 18464 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:31:43.041659 18464 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:31:43.041811 18464 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:31:43.041901 18464 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:31:43.077507 18464 rpc_server.cc:307] RPC server started. Bound to: 127.8.18.193:33097
I20260810 11:31:43.077556 18593 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.18.193:33097 every 8 connection(s)
I20260810 11:31:43.079017 18464 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786361401922362-8267-0/minicluster-data/ts-0/data/info.pb
I20260810 11:31:43.083674 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 18464
I20260810 11:31:43.083922 8267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786361401922362-8267-0/minicluster-data/ts-0/wal/instance
I20260810 11:31:43.084671 8267 external_mini_cluster.cc:1428] Setting key 2339ce15a33783d6f8c2a71c9d24bf6c
I20260810 11:31:43.088496 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786361401922362-8267-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786361401922362-8267-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786361401922362-8267-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786361401922362-8267-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.18.194:0
--local_ip_for_outbound_sockets=127.8.18.194
--webserver_interface=127.8.18.194
--webserver_port=0
--tserver_master_addrs=127.8.18.254:45303
--builtin_ntp_servers=127.8.18.212:41147
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260810 11:31:43.089684 18596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:43.082158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.193:60831 (local address 127.8.18.254:45303)
0810 11:31:43.082677 (+ 519us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:43.082682 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:43.084387 (+ 1705us) server_negotiation.cc:407] Connection header received
0810 11:31:43.086731 (+ 2344us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:43.086773 (+ 42us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:43.086852 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:43.086976 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:31:43.088758 (+ 1782us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:31:43.088769 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:31:43.088772 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:31:43.088792 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:31:43.088839 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:31:43.088852 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:31:43.089010 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0810 11:31:43.089223 (+ 213us) server_negotiation.cc:299] Negotiation successful
0810 11:31:43.089304 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":398,"thread_start_us":260,"threads_started":1}
I20260810 11:31:43.090792 18597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:43.082894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:45303 (local address 127.8.18.193:60831)
0810 11:31:43.084175 (+ 1281us) negotiation.cc:107] Waiting for socket to connect
0810 11:31:43.084235 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0810 11:31:43.086341 (+ 2106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:31:43.087317 (+ 976us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:31:43.087340 (+ 23us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:31:43.087464 (+ 124us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:31:43.088369 (+ 905us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 11:31:43.088372 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:31:43.088418 (+ 46us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 11:31:43.088529 (+ 111us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:31:43.088964 (+ 435us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:31:43.088978 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:31:43.088988 (+ 10us) client_negotiation.cc:769] Sending connection context
0810 11:31:43.089318 (+ 330us) client_negotiation.cc:240] Negotiation successful
0810 11:31:43.089678 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":653,"thread_start_us":430,"threads_started":1}
I20260810 11:31:43.092782 18594 heartbeater.cc:347] Connected to a master server at 127.8.18.254:45303
I20260810 11:31:43.093163 18594 heartbeater.cc:464] Registering TS with master...
I20260810 11:31:43.093946 18594 heartbeater.cc:511] Master 127.8.18.254:45303 requested a full tablet report, sending...
I20260810 11:31:43.096618 18402 ts_manager.cc:194] Registered new tserver with Master: 3d5cda76cb1f4241aacfe6c4a8d32672 (127.8.18.193:33097)
W20260810 11:31:43.448056 18598 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:31:43.448501 18598 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:31:43.448691 18598 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:31:43.456712 18598 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:31:43.456912 18598 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:31:43.457139 18598 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.18.194
I20260810 11:31:43.471303 18598 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.18.212:41147
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786361401922362-8267-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786361401922362-8267-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.18.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786361401922362-8267-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.18.194
--webserver_port=0
--tserver_master_addrs=127.8.18.254:45303
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.18.194
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786361401922362-8267-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:31:43.473042 18598 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:31:43.475418 18598 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:31:43.492344 18604 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:31:43.492319 18603 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:31:43.492761 18606 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:31:43.493134 18598 server_base.cc:1034] running on GCE node
I20260810 11:31:43.493955 18598 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:31:43.495117 18598 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:31:43.496402 18598 hybrid_clock.cc:648] HybridClock initialized: now 1786361503496361 us; error 58 us; skew 500 ppm
I20260810 11:31:43.496826 18598 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 11:31:43.503450 18598 webserver.cc:460] Webserver started at http://127.8.18.194:37031/ using document root <none> and password file <none>
I20260810 11:31:43.504416 18598 fs_manager.cc:359] Metadata directory not provided
I20260810 11:31:43.504593 18598 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:31:43.505026 18598 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:31:43.508519 18598 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786361401922362-8267-0/minicluster-data/ts-1/data/instance:
uuid: "d5156095abbf4ab1981f0737979f9eea"
format_stamp: "Formatted at 2026-08-10 11:31:43 on dist-test-slave-srgv"
server_key: "8def2ea24f34649e6347e86ff4d47342"
server_key_iv: "3b5e5cc230dedc8fa8ddf967992f9179"
server_key_version: "encryptionkey@0"
I20260810 11:31:43.509606 18598 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786361401922362-8267-0/minicluster-data/ts-1/wal/instance:
uuid: "d5156095abbf4ab1981f0737979f9eea"
format_stamp: "Formatted at 2026-08-10 11:31:43 on dist-test-slave-srgv"
server_key: "8def2ea24f34649e6347e86ff4d47342"
server_key_iv: "3b5e5cc230dedc8fa8ddf967992f9179"
server_key_version: "encryptionkey@0"
I20260810 11:31:43.515949 18598 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260810 11:31:43.520220 18612 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:31:43.521963 18598 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.001s
I20260810 11:31:43.522275 18598 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786361401922362-8267-0/minicluster-data/ts-1/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786361401922362-8267-0/minicluster-data/ts-1/wal
uuid: "d5156095abbf4ab1981f0737979f9eea"
format_stamp: "Formatted at 2026-08-10 11:31:43 on dist-test-slave-srgv"
server_key: "8def2ea24f34649e6347e86ff4d47342"
server_key_iv: "3b5e5cc230dedc8fa8ddf967992f9179"
server_key_version: "encryptionkey@0"
I20260810 11:31:43.522544 18598 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786361401922362-8267-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786361401922362-8267-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786361401922362-8267-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:31:43.529150 18598 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:31:43.530197 18598 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:31:43.530524 18598 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:31:43.531633 18598 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:31:43.533267 18598 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:31:43.533440 18598 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:31:43.533604 18598 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:31:43.533706 18598 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:31:43.570236 18598 rpc_server.cc:307] RPC server started. Bound to: 127.8.18.194:36187
I20260810 11:31:43.570343 18730 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.18.194:36187 every 8 connection(s)
I20260810 11:31:43.571681 18598 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786361401922362-8267-0/minicluster-data/ts-1/data/info.pb
I20260810 11:31:43.580824 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 18598
I20260810 11:31:43.581142 8267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786361401922362-8267-0/minicluster-data/ts-1/wal/instance
I20260810 11:31:43.581210 18596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:43.574452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.194:36617 (local address 127.8.18.254:45303)
0810 11:31:43.574709 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:43.574716 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:43.576373 (+ 1657us) server_negotiation.cc:407] Connection header received
0810 11:31:43.578504 (+ 2131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:43.578509 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:43.578579 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:43.578699 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:31:43.580320 (+ 1621us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:31:43.580325 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:31:43.580328 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:31:43.580348 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:31:43.580385 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:31:43.580398 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:31:43.580600 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0810 11:31:43.580819 (+ 219us) server_negotiation.cc:299] Negotiation successful
0810 11:31:43.580907 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260810 11:31:43.581748 8267 external_mini_cluster.cc:1428] Setting key a7c50488651e4eb4496dc245defe5968
I20260810 11:31:43.582638 18733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:43.575067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:45303 (local address 127.8.18.194:36617)
0810 11:31:43.576151 (+ 1084us) negotiation.cc:107] Waiting for socket to connect
0810 11:31:43.576202 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0810 11:31:43.578136 (+ 1934us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:31:43.579036 (+ 900us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:31:43.579053 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:31:43.579167 (+ 114us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:31:43.579917 (+ 750us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 11:31:43.579920 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:31:43.579965 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 11:31:43.580111 (+ 146us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:31:43.580543 (+ 432us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:31:43.580558 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:31:43.580569 (+ 11us) client_negotiation.cc:769] Sending connection context
0810 11:31:43.580922 (+ 353us) client_negotiation.cc:240] Negotiation successful
0810 11:31:43.581371 (+ 449us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":540,"thread_start_us":317,"threads_started":1}
I20260810 11:31:43.584349 18731 heartbeater.cc:347] Connected to a master server at 127.8.18.254:45303
I20260810 11:31:43.584749 18731 heartbeater.cc:464] Registering TS with master...
I20260810 11:31:43.585393 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786361401922362-8267-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786361401922362-8267-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786361401922362-8267-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786361401922362-8267-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.18.195:0
--local_ip_for_outbound_sockets=127.8.18.195
--webserver_interface=127.8.18.195
--webserver_port=0
--tserver_master_addrs=127.8.18.254:45303
--builtin_ntp_servers=127.8.18.212:41147
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260810 11:31:43.585595 18731 heartbeater.cc:511] Master 127.8.18.254:45303 requested a full tablet report, sending...
I20260810 11:31:43.587476 18402 ts_manager.cc:194] Registered new tserver with Master: d5156095abbf4ab1981f0737979f9eea (127.8.18.194:36187)
W20260810 11:31:43.943545 18734 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:31:43.944108 18734 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:31:43.944291 18734 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:31:43.956048 18734 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:31:43.956185 18734 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:31:43.956377 18734 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.18.195
I20260810 11:31:43.966570 18734 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.18.212:41147
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786361401922362-8267-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786361401922362-8267-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.18.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786361401922362-8267-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.18.195
--webserver_port=0
--tserver_master_addrs=127.8.18.254:45303
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.18.195
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786361401922362-8267-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:31:43.967993 18734 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:31:43.970031 18734 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:31:43.984840 18739 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:31:43.984913 18740 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:31:43.984890 18742 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:31:43.985857 18734 server_base.cc:1034] running on GCE node
I20260810 11:31:43.986616 18734 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:31:43.987757 18734 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:31:43.989040 18734 hybrid_clock.cc:648] HybridClock initialized: now 1786361503988988 us; error 54 us; skew 500 ppm
I20260810 11:31:43.989568 18734 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 11:31:43.992219 18734 webserver.cc:460] Webserver started at http://127.8.18.195:38961/ using document root <none> and password file <none>
I20260810 11:31:43.993062 18734 fs_manager.cc:359] Metadata directory not provided
I20260810 11:31:43.993227 18734 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:31:43.993724 18734 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:31:43.997041 18734 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786361401922362-8267-0/minicluster-data/ts-2/data/instance:
uuid: "dc130ccf3fb445d9bcde32bd06155445"
format_stamp: "Formatted at 2026-08-10 11:31:43 on dist-test-slave-srgv"
server_key: "13599ba3eb2c6851c302e200123fae21"
server_key_iv: "98a0e6ef9e0ed99140665843b92c8c0d"
server_key_version: "encryptionkey@0"
I20260810 11:31:43.998080 18734 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786361401922362-8267-0/minicluster-data/ts-2/wal/instance:
uuid: "dc130ccf3fb445d9bcde32bd06155445"
format_stamp: "Formatted at 2026-08-10 11:31:43 on dist-test-slave-srgv"
server_key: "13599ba3eb2c6851c302e200123fae21"
server_key_iv: "98a0e6ef9e0ed99140665843b92c8c0d"
server_key_version: "encryptionkey@0"
I20260810 11:31:44.004376 18734 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260810 11:31:44.008911 18749 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:31:44.010756 18734 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260810 11:31:44.010977 18734 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786361401922362-8267-0/minicluster-data/ts-2/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786361401922362-8267-0/minicluster-data/ts-2/wal
uuid: "dc130ccf3fb445d9bcde32bd06155445"
format_stamp: "Formatted at 2026-08-10 11:31:43 on dist-test-slave-srgv"
server_key: "13599ba3eb2c6851c302e200123fae21"
server_key_iv: "98a0e6ef9e0ed99140665843b92c8c0d"
server_key_version: "encryptionkey@0"
I20260810 11:31:44.011252 18734 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786361401922362-8267-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786361401922362-8267-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786361401922362-8267-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:31:44.018124 18734 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:31:44.019090 18734 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:31:44.019492 18734 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:31:44.020538 18734 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:31:44.022162 18734 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:31:44.022254 18734 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:31:44.022343 18734 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:31:44.022398 18734 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:31:44.059348 18734 rpc_server.cc:307] RPC server started. Bound to: 127.8.18.195:33517
I20260810 11:31:44.059408 18865 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.18.195:33517 every 8 connection(s)
I20260810 11:31:44.060864 18734 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786361401922362-8267-0/minicluster-data/ts-2/data/info.pb
I20260810 11:31:44.065536 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 18734
I20260810 11:31:44.065739 8267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786361401922362-8267-0/minicluster-data/ts-2/wal/instance
I20260810 11:31:44.066416 8267 external_mini_cluster.cc:1428] Setting key 3973b189c106427be928c82a3815840b
I20260810 11:31:44.071008 18596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:44.063928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.195:57485 (local address 127.8.18.254:45303)
0810 11:31:44.064150 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:44.064155 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:44.066136 (+ 1981us) server_negotiation.cc:407] Connection header received
0810 11:31:44.068378 (+ 2242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:44.068389 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:44.068453 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:44.068599 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:31:44.070264 (+ 1665us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:31:44.070271 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:31:44.070276 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:31:44.070306 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:31:44.070354 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:31:44.070371 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:31:44.070472 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0810 11:31:44.070712 (+ 240us) server_negotiation.cc:299] Negotiation successful
0810 11:31:44.070774 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260810 11:31:44.072167 18868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:44.064489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:45303 (local address 127.8.18.195:57485)
0810 11:31:44.065920 (+ 1431us) negotiation.cc:107] Waiting for socket to connect
0810 11:31:44.065978 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0810 11:31:44.067891 (+ 1913us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:31:44.068840 (+ 949us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:31:44.068849 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:31:44.068956 (+ 107us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:31:44.069843 (+ 887us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 11:31:44.069847 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:31:44.069921 (+ 74us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 11:31:44.070055 (+ 134us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:31:44.070487 (+ 432us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:31:44.070498 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:31:44.070508 (+ 10us) client_negotiation.cc:769] Sending connection context
0810 11:31:44.070792 (+ 284us) client_negotiation.cc:240] Negotiation successful
0810 11:31:44.071033 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":736,"thread_start_us":486,"threads_started":1}
I20260810 11:31:44.073839 18866 heartbeater.cc:347] Connected to a master server at 127.8.18.254:45303
I20260810 11:31:44.074159 18866 heartbeater.cc:464] Registering TS with master...
I20260810 11:31:44.074813 18866 heartbeater.cc:511] Master 127.8.18.254:45303 requested a full tablet report, sending...
I20260810 11:31:44.076552 18402 ts_manager.cc:194] Registered new tserver with Master: dc130ccf3fb445d9bcde32bd06155445 (127.8.18.195:33517)
I20260810 11:31:44.083606 8267 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260810 11:31:44.091243 18596 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:31:44.089094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51624 (local address 127.8.18.254:45303)
0810 11:31:44.089374 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:44.089383 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:44.089977 (+ 594us) server_negotiation.cc:407] Connection header received
0810 11:31:44.090232 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:44.090237 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:44.090300 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:44.090418 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:31:44.090839 (+ 421us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51624: BlockingRecv error: recv got EOF from 127.0.0.1:51624 (error 108)
Metrics: {"server-negotiator.queue_time_us":105}
I20260810 11:31:44.092193 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 18464
I20260810 11:31:44.096222 18875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:44.093409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34174 (local address 127.8.18.193:33097)
0810 11:31:44.094048 (+ 639us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:44.094056 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:44.094085 (+ 29us) server_negotiation.cc:407] Connection header received
0810 11:31:44.094157 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:44.094165 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:44.094398 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:44.094538 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:31:44.095508 (+ 970us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:31:44.095515 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:31:44.095527 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:31:44.095542 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:31:44.095592 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:31:44.095607 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:31:44.095777 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0810 11:31:44.095841 (+ 64us) server_negotiation.cc:299] Negotiation successful
0810 11:31:44.095923 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":491,"thread_start_us":327,"threads_started":1}
I20260810 11:31:44.097664 18589 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:31:44.100945 18594 heartbeater.cc:503] Master 127.8.18.254:45303 was elected leader, sending a full tablet report...
I20260810 11:31:44.183431 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 18464
I20260810 11:31:44.202442 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 18598
I20260810 11:31:44.206619 18877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:44.203545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51394 (local address 127.8.18.194:36187)
0810 11:31:44.204110 (+ 565us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:44.204118 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:44.204147 (+ 29us) server_negotiation.cc:407] Connection header received
0810 11:31:44.204215 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:44.204221 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:44.204450 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:44.204638 (+ 188us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:31:44.205835 (+ 1197us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:31:44.205843 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:31:44.205855 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:31:44.205870 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:31:44.205923 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:31:44.205943 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:31:44.206109 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0810 11:31:44.206209 (+ 100us) server_negotiation.cc:299] Negotiation successful
0810 11:31:44.206335 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":397,"thread_start_us":241,"threads_started":1}
I20260810 11:31:44.208230 18726 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:31:44.292759 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 18598
I20260810 11:31:44.311594 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 18734
I20260810 11:31:44.316231 18880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:44.312732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48752 (local address 127.8.18.195:33517)
0810 11:31:44.313400 (+ 668us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:44.313460 (+ 60us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:44.313517 (+ 57us) server_negotiation.cc:407] Connection header received
0810 11:31:44.313593 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:44.313600 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:44.313839 (+ 239us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:44.314026 (+ 187us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:31:44.315363 (+ 1337us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:31:44.315371 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:31:44.315383 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:31:44.315402 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:31:44.315465 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:31:44.315480 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:31:44.315702 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0810 11:31:44.315805 (+ 103us) server_negotiation.cc:299] Negotiation successful
0810 11:31:44.315920 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":516,"thread_start_us":333,"threads_started":1}
I20260810 11:31:44.317929 18861 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:31:44.403082 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 18734
I20260810 11:31:44.421522 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 18371
I20260810 11:31:44.422958 18434 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:31:44.533563 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 18371
2026-08-10T11:31:44Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (2625 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786361401922362-8267-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 11:31:44 dist-test-slave-srgv krb5kdc[18885](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 11:31:44 dist-test-slave-srgv krb5kdc[18885](info): set up 2 sockets
Aug 10 11:31:44 dist-test-slave-srgv krb5kdc[18885](info): commencing operation
krb5kdc: starting...
W20260810 11:31:45.678231 8267 mini_kdc.cc:121] Time spent starting KDC: real 1.102s user 0.003s sys 0.031s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786361401922362-8267-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786361401922362-8267-0/krb5kdc/test-user.keytab.
Aug 10 11:31:45 dist-test-slave-srgv krb5kdc[18885](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361505, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T11:31:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T11:31:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.18.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.18.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.18.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.254.keytab.
Entry for principal kudu/127.8.18.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.254.keytab.
I20260810 11:31:45.975248 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786361401922362-8267-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786361401922362-8267-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786361401922362-8267-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.18.254:33599
--webserver_interface=127.8.18.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.254.keytab
--principal=kudu/127.8.18.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.18.212:33827
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.18.254:33599
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786361401922362-8267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786361401922362-8267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786361401922362-8267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:31:46.344901 18898 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:31:46.345259 18898 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:31:46.345359 18898 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:31:46.353379 18898 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 11:31:46.353590 18898 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:31:46.353644 18898 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:31:46.353705 18898 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 11:31:46.353749 18898 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 11:31:46.364537 18898 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.18.212:33827
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786361401922362-8267-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786361401922362-8267-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.18.254:33599
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.254.keytab
--principal=kudu/127.8.18.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.18.254:33599
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.18.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786361401922362-8267-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:31:46.366061 18898 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:31:46.367386 18898 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:31:46.380543 18906 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:31:46.380470 18904 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:31:46.380589 18903 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:31:46.381708 18898 server_base.cc:1034] running on GCE node
I20260810 11:31:46.382614 18898 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:31:46.384290 18898 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:31:46.385550 18898 hybrid_clock.cc:648] HybridClock initialized: now 1786361506385498 us; error 44 us; skew 500 ppm
I20260810 11:31:46.385968 18898 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:31:46 dist-test-slave-srgv krb5kdc[18885](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361506, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:31:46.389746 18898 init.cc:377] Logged in from keytab as kudu/127.8.18.254@KRBTEST.COM (short username kudu)
I20260810 11:31:46.391760 18898 webserver.cc:460] Webserver started at http://127.8.18.254:37987/ using document root <none> and password file <none>
I20260810 11:31:46.392606 18898 fs_manager.cc:359] Metadata directory not provided
I20260810 11:31:46.393116 18898 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:31:46.393635 18898 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:31:46.397126 18898 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786361401922362-8267-0/minicluster-data/master-0/data/instance:
uuid: "1cd6f5420b964bdc8502f9bb2456d0b9"
format_stamp: "Formatted at 2026-08-10 11:31:46 on dist-test-slave-srgv"
server_key: "c7ab98dd05c4675ff8817d3c20567b6f"
server_key_iv: "927eca8fb696874da25da49575d563db"
server_key_version: "encryptionkey@0"
I20260810 11:31:46.398103 18898 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786361401922362-8267-0/minicluster-data/master-0/wal/instance:
uuid: "1cd6f5420b964bdc8502f9bb2456d0b9"
format_stamp: "Formatted at 2026-08-10 11:31:46 on dist-test-slave-srgv"
server_key: "c7ab98dd05c4675ff8817d3c20567b6f"
server_key_iv: "927eca8fb696874da25da49575d563db"
server_key_version: "encryptionkey@0"
I20260810 11:31:46.404309 18898 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260810 11:31:46.408437 18913 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:31:46.410315 18898 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260810 11:31:46.410575 18898 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786361401922362-8267-0/minicluster-data/master-0/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786361401922362-8267-0/minicluster-data/master-0/wal
uuid: "1cd6f5420b964bdc8502f9bb2456d0b9"
format_stamp: "Formatted at 2026-08-10 11:31:46 on dist-test-slave-srgv"
server_key: "c7ab98dd05c4675ff8817d3c20567b6f"
server_key_iv: "927eca8fb696874da25da49575d563db"
server_key_version: "encryptionkey@0"
I20260810 11:31:46.410804 18898 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786361401922362-8267-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786361401922362-8267-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786361401922362-8267-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:31:46.435552 18898 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:31:46.440634 18898 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:31:46.441087 18898 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:31:46.463573 18898 rpc_server.cc:307] RPC server started. Bound to: 127.8.18.254:33599
I20260810 11:31:46.463634 18966 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.18.254:33599 every 8 connection(s)
I20260810 11:31:46.465106 18898 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
I20260810 11:31:46.469165 18967 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:31:46.475590 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 18898
I20260810 11:31:46.475845 8267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786361401922362-8267-0/minicluster-data/master-0/wal/instance
I20260810 11:31:46.476781 8267 external_mini_cluster.cc:1428] Setting key ed81b2f72fee4d75d2ab57160a7c5145
I20260810 11:31:46.483276 18967 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1cd6f5420b964bdc8502f9bb2456d0b9: Bootstrap starting.
I20260810 11:31:46.487116 18967 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1cd6f5420b964bdc8502f9bb2456d0b9: Neither blocks nor log segments found. Creating new log.
I20260810 11:31:46.488380 18967 log.cc:824] T 00000000000000000000000000000000 P 1cd6f5420b964bdc8502f9bb2456d0b9: Log is configured to *not* fsync() on all Append() calls
I20260810 11:31:46.491801 18967 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1cd6f5420b964bdc8502f9bb2456d0b9: No bootstrap required, opened a new log
Aug 10 11:31:46 dist-test-slave-srgv krb5kdc[18885](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361505, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.18.254@KRBTEST.COM
I20260810 11:31:46.501626 18967 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1cd6f5420b964bdc8502f9bb2456d0b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1cd6f5420b964bdc8502f9bb2456d0b9" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 33599 } }
I20260810 11:31:46.502207 18967 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1cd6f5420b964bdc8502f9bb2456d0b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:31:46.502348 18967 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1cd6f5420b964bdc8502f9bb2456d0b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1cd6f5420b964bdc8502f9bb2456d0b9, State: Initialized, Role: FOLLOWER
I20260810 11:31:46.503079 18967 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1cd6f5420b964bdc8502f9bb2456d0b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1cd6f5420b964bdc8502f9bb2456d0b9" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 33599 } }
I20260810 11:31:46.503358 18967 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1cd6f5420b964bdc8502f9bb2456d0b9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 11:31:46.503516 18967 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1cd6f5420b964bdc8502f9bb2456d0b9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 11:31:46.503731 18967 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1cd6f5420b964bdc8502f9bb2456d0b9 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:31:46.504783 18970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:46.479998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44992 (local address 127.8.18.254:33599)
0810 11:31:46.481007 (+ 1009us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:46.481020 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:46.481069 (+ 49us) server_negotiation.cc:407] Connection header received
0810 11:31:46.483012 (+ 1943us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:46.483040 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:46.483602 (+ 562us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:46.484116 (+ 514us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:31:46.485978 (+ 1862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:46.487467 (+ 1489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:46.489023 (+ 1556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:46.489898 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:46.490122 (+ 224us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:46.495034 (+ 4912us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:31:46.495043 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:31:46.495064 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:31:46.495112 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:31:46.498822 (+ 3710us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:31:46.499959 (+ 1137us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:31:46.499968 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:31:46.499975 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:31:46.500268 (+ 293us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:31:46.500788 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:31:46.500792 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:31:46.500794 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:31:46.501628 (+ 834us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:31:46.501888 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0810 11:31:46.502713 (+ 825us) server_negotiation.cc:299] Negotiation successful
0810 11:31:46.503234 (+ 521us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":742,"thread_start_us":460,"threads_started":1}
I20260810 11:31:46.506894 18967 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1cd6f5420b964bdc8502f9bb2456d0b9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1cd6f5420b964bdc8502f9bb2456d0b9" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 33599 } }
I20260810 11:31:46.507575 18967 leader_election.cc:302] T 00000000000000000000000000000000 P 1cd6f5420b964bdc8502f9bb2456d0b9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1cd6f5420b964bdc8502f9bb2456d0b9; no voters:
I20260810 11:31:46.508343 18967 leader_election.cc:288] T 00000000000000000000000000000000 P 1cd6f5420b964bdc8502f9bb2456d0b9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 11:31:46.508538 18972 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1cd6f5420b964bdc8502f9bb2456d0b9 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:31:46.509506 18972 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1cd6f5420b964bdc8502f9bb2456d0b9 [term 1 LEADER]: Becoming Leader. State: Replica: 1cd6f5420b964bdc8502f9bb2456d0b9, State: Running, Role: LEADER
I20260810 11:31:46.510123 18972 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1cd6f5420b964bdc8502f9bb2456d0b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1cd6f5420b964bdc8502f9bb2456d0b9" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 33599 } }
I20260810 11:31:46.510944 18967 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1cd6f5420b964bdc8502f9bb2456d0b9 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 11:31:46.512989 18973 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1cd6f5420b964bdc8502f9bb2456d0b9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1cd6f5420b964bdc8502f9bb2456d0b9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1cd6f5420b964bdc8502f9bb2456d0b9" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 33599 } } }
I20260810 11:31:46.513401 18973 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1cd6f5420b964bdc8502f9bb2456d0b9 [sys.catalog]: This master's current role is: LEADER
I20260810 11:31:46.514302 18974 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1cd6f5420b964bdc8502f9bb2456d0b9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1cd6f5420b964bdc8502f9bb2456d0b9. Latest consensus state: current_term: 1 leader_uuid: "1cd6f5420b964bdc8502f9bb2456d0b9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1cd6f5420b964bdc8502f9bb2456d0b9" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 33599 } } }
I20260810 11:31:46.515118 18974 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1cd6f5420b964bdc8502f9bb2456d0b9 [sys.catalog]: This master's current role is: LEADER
I20260810 11:31:46.516633 18977 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 11:31:46.522990 18977 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 11:31:46.533193 18977 catalog_manager.cc:1347] Generated new cluster ID: 301d55e8099145ffb07ee9a898f1f6fa
I20260810 11:31:46.533350 18977 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 11:31:46.556643 18977 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 11:31:46.558682 18977 catalog_manager.cc:1504] Loading token signing keys...
I20260810 11:31:46.568512 18977 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1cd6f5420b964bdc8502f9bb2456d0b9: Generated new TSK 0
I20260810 11:31:46.569748 18977 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.18.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.18.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.18.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.193.keytab.
Entry for principal kudu/127.8.18.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.193.keytab.
I20260810 11:31:46.643781 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786361401922362-8267-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786361401922362-8267-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786361401922362-8267-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786361401922362-8267-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.18.193:0
--local_ip_for_outbound_sockets=127.8.18.193
--webserver_interface=127.8.18.193
--webserver_port=0
--tserver_master_addrs=127.8.18.254:33599
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.193.keytab
--principal=kudu/127.8.18.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.18.212:33827
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786361401922362-8267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786361401922362-8267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786361401922362-8267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:31:47.028425 18993 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:31:47.028836 18993 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:31:47.029026 18993 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:31:47.037732 18993 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:31:47.037914 18993 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:31:47.038113 18993 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.18.193
I20260810 11:31:47.049839 18993 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.18.212:33827
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786361401922362-8267-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786361401922362-8267-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.193.keytab
--principal=kudu/127.8.18.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.18.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786361401922362-8267-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.18.193
--webserver_port=0
--tserver_master_addrs=127.8.18.254:33599
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.18.193
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786361401922362-8267-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:31:47.051463 18993 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:31:47.053072 18993 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:31:47.070130 18998 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:31:47.070112 18999 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:31:47.070801 19001 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:31:47.071201 18993 server_base.cc:1034] running on GCE node
I20260810 11:31:47.071900 18993 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:31:47.073045 18993 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:31:47.074249 18993 hybrid_clock.cc:648] HybridClock initialized: now 1786361507074171 us; error 74 us; skew 500 ppm
I20260810 11:31:47.074641 18993 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:31:47 dist-test-slave-srgv krb5kdc[18885](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361507, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:31:47.078249 18993 init.cc:377] Logged in from keytab as kudu/127.8.18.193@KRBTEST.COM (short username kudu)
I20260810 11:31:47.079916 18993 webserver.cc:460] Webserver started at http://127.8.18.193:36853/ using document root <none> and password file <none>
I20260810 11:31:47.080677 18993 fs_manager.cc:359] Metadata directory not provided
I20260810 11:31:47.080837 18993 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:31:47.081168 18993 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:31:47.084438 18993 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786361401922362-8267-0/minicluster-data/ts-0/data/instance:
uuid: "95b8b02d46dc4bcf9efa75832f4b0cbf"
format_stamp: "Formatted at 2026-08-10 11:31:47 on dist-test-slave-srgv"
server_key: "d7ba97f75cf8e403c2d79c9fd8b8fef0"
server_key_iv: "a3beae1a7152a47f7f5ad0c1e5823221"
server_key_version: "encryptionkey@0"
I20260810 11:31:47.085366 18993 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786361401922362-8267-0/minicluster-data/ts-0/wal/instance:
uuid: "95b8b02d46dc4bcf9efa75832f4b0cbf"
format_stamp: "Formatted at 2026-08-10 11:31:47 on dist-test-slave-srgv"
server_key: "d7ba97f75cf8e403c2d79c9fd8b8fef0"
server_key_iv: "a3beae1a7152a47f7f5ad0c1e5823221"
server_key_version: "encryptionkey@0"
I20260810 11:31:47.091005 18993 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260810 11:31:47.094825 19008 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:31:47.096248 18993 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 11:31:47.096454 18993 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786361401922362-8267-0/minicluster-data/ts-0/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786361401922362-8267-0/minicluster-data/ts-0/wal
uuid: "95b8b02d46dc4bcf9efa75832f4b0cbf"
format_stamp: "Formatted at 2026-08-10 11:31:47 on dist-test-slave-srgv"
server_key: "d7ba97f75cf8e403c2d79c9fd8b8fef0"
server_key_iv: "a3beae1a7152a47f7f5ad0c1e5823221"
server_key_version: "encryptionkey@0"
I20260810 11:31:47.096696 18993 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786361401922362-8267-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786361401922362-8267-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786361401922362-8267-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:31:47.114512 18993 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:31:47.119588 18993 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:31:47.119918 18993 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:31:47.120982 18993 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:31:47.122460 18993 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:31:47.122570 18993 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:31:47.122742 18993 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:31:47.122843 18993 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:31:47.158699 18993 rpc_server.cc:307] RPC server started. Bound to: 127.8.18.193:41165
I20260810 11:31:47.158763 19123 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.18.193:41165 every 8 connection(s)
I20260810 11:31:47.160171 18993 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786361401922362-8267-0/minicluster-data/ts-0/data/info.pb
I20260810 11:31:47.166334 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 18993
I20260810 11:31:47.166546 8267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786361401922362-8267-0/minicluster-data/ts-0/wal/instance
I20260810 11:31:47.167263 8267 external_mini_cluster.cc:1428] Setting key fd90bddd76d2ce29e8fdb6b5f292d4da
Aug 10 11:31:47 dist-test-slave-srgv krb5kdc[18885](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361507, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.193@KRBTEST.COM for kudu/127.8.18.254@KRBTEST.COM
I20260810 11:31:47.185925 19126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:47.163029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.193:37691 (local address 127.8.18.254:33599)
0810 11:31:47.163501 (+ 472us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:47.163505 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:47.164722 (+ 1217us) server_negotiation.cc:407] Connection header received
0810 11:31:47.167204 (+ 2482us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:47.167210 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:47.167309 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:47.167434 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:31:47.169721 (+ 2287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:47.170463 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:47.172174 (+ 1711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:47.172747 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:47.172898 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:47.178120 (+ 5222us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:31:47.178129 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:31:47.178136 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:31:47.178185 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:31:47.181961 (+ 3776us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:31:47.182868 (+ 907us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:31:47.182877 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:31:47.182883 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:31:47.183073 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:31:47.183565 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:31:47.183573 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:31:47.183579 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:31:47.183999 (+ 420us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:31:47.184210 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0810 11:31:47.185060 (+ 850us) server_negotiation.cc:299] Negotiation successful
0810 11:31:47.185363 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":352,"thread_start_us":236,"threads_started":1}
I20260810 11:31:47.186753 19127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:47.163629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:33599 (local address 127.8.18.193:37691)
0810 11:31:47.164488 (+ 859us) negotiation.cc:107] Waiting for socket to connect
0810 11:31:47.164534 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0810 11:31:47.166850 (+ 2316us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:31:47.167713 (+ 863us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:31:47.167724 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:31:47.168552 (+ 828us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:31:47.169514 (+ 962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:47.169546 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:47.170673 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:47.170679 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:47.171982 (+ 1303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:47.171995 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:47.173009 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:47.173017 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:47.173287 (+ 270us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:47.174152 (+ 865us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:31:47.174179 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:31:47.177804 (+ 3625us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:31:47.182161 (+ 4357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:31:47.182171 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:31:47.182186 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:31:47.182691 (+ 505us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:31:47.183196 (+ 505us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:31:47.183200 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:31:47.183202 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:31:47.183428 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:31:47.184196 (+ 768us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:31:47.184204 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:31:47.184594 (+ 390us) client_negotiation.cc:769] Sending connection context
0810 11:31:47.184863 (+ 269us) client_negotiation.cc:240] Negotiation successful
0810 11:31:47.185332 (+ 469us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":593,"thread_start_us":360,"threads_started":1}
I20260810 11:31:47.188665 19124 heartbeater.cc:347] Connected to a master server at 127.8.18.254:33599
I20260810 11:31:47.189090 19124 heartbeater.cc:464] Registering TS with master...
I20260810 11:31:47.190184 19124 heartbeater.cc:511] Master 127.8.18.254:33599 requested a full tablet report, sending...
I20260810 11:31:47.192891 18932 ts_manager.cc:194] Registered new tserver with Master: 95b8b02d46dc4bcf9efa75832f4b0cbf (127.8.18.193:41165)
I20260810 11:31:47.194583 18932 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.18.193@KRBTEST.COM'} at 127.8.18.193:37691
WARNING: no policy specified for kudu/127.8.18.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.18.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.18.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.194.keytab.
Entry for principal kudu/127.8.18.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.194.keytab.
I20260810 11:31:47.233669 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786361401922362-8267-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786361401922362-8267-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786361401922362-8267-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786361401922362-8267-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.18.194:0
--local_ip_for_outbound_sockets=127.8.18.194
--webserver_interface=127.8.18.194
--webserver_port=0
--tserver_master_addrs=127.8.18.254:33599
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.194.keytab
--principal=kudu/127.8.18.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.18.212:33827
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786361401922362-8267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786361401922362-8267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786361401922362-8267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:31:47.598755 19130 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:31:47.599206 19130 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:31:47.599387 19130 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:31:47.607084 19130 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:31:47.607276 19130 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:31:47.607474 19130 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.18.194
I20260810 11:31:47.618798 19130 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.18.212:33827
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786361401922362-8267-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786361401922362-8267-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.194.keytab
--principal=kudu/127.8.18.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.18.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786361401922362-8267-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.18.194
--webserver_port=0
--tserver_master_addrs=127.8.18.254:33599
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.18.194
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786361401922362-8267-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:31:47.620299 19130 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:31:47.621892 19130 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:31:47.636745 19135 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:31:47.636786 19138 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:31:47.636752 19136 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:31:47.637526 19130 server_base.cc:1034] running on GCE node
I20260810 11:31:47.638262 19130 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:31:47.639343 19130 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:31:47.640595 19130 hybrid_clock.cc:648] HybridClock initialized: now 1786361507640536 us; error 59 us; skew 500 ppm
I20260810 11:31:47.640964 19130 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:31:47 dist-test-slave-srgv krb5kdc[18885](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361507, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:31:47.644573 19130 init.cc:377] Logged in from keytab as kudu/127.8.18.194@KRBTEST.COM (short username kudu)
I20260810 11:31:47.646335 19130 webserver.cc:460] Webserver started at http://127.8.18.194:39987/ using document root <none> and password file <none>
I20260810 11:31:47.647082 19130 fs_manager.cc:359] Metadata directory not provided
I20260810 11:31:47.647199 19130 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:31:47.647490 19130 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:31:47.650780 19130 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786361401922362-8267-0/minicluster-data/ts-1/data/instance:
uuid: "7e7c8d1f0b0c4ac3af00f2bb1c85b15f"
format_stamp: "Formatted at 2026-08-10 11:31:47 on dist-test-slave-srgv"
server_key: "63d6475ce284f8f4dd5ec7ca4e52d78d"
server_key_iv: "5fe7d3671dc5e8721672a1ba433baeab"
server_key_version: "encryptionkey@0"
I20260810 11:31:47.651657 19130 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786361401922362-8267-0/minicluster-data/ts-1/wal/instance:
uuid: "7e7c8d1f0b0c4ac3af00f2bb1c85b15f"
format_stamp: "Formatted at 2026-08-10 11:31:47 on dist-test-slave-srgv"
server_key: "63d6475ce284f8f4dd5ec7ca4e52d78d"
server_key_iv: "5fe7d3671dc5e8721672a1ba433baeab"
server_key_version: "encryptionkey@0"
I20260810 11:31:47.658177 19130 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.003s
I20260810 11:31:47.662278 19145 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:31:47.664063 19130 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.003s
I20260810 11:31:47.664242 19130 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786361401922362-8267-0/minicluster-data/ts-1/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786361401922362-8267-0/minicluster-data/ts-1/wal
uuid: "7e7c8d1f0b0c4ac3af00f2bb1c85b15f"
format_stamp: "Formatted at 2026-08-10 11:31:47 on dist-test-slave-srgv"
server_key: "63d6475ce284f8f4dd5ec7ca4e52d78d"
server_key_iv: "5fe7d3671dc5e8721672a1ba433baeab"
server_key_version: "encryptionkey@0"
I20260810 11:31:47.664433 19130 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786361401922362-8267-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786361401922362-8267-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786361401922362-8267-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:31:47.695173 19130 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:31:47.700325 19130 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:31:47.700618 19130 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:31:47.701836 19130 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:31:47.703451 19130 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:31:47.703548 19130 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:31:47.703708 19130 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:31:47.703771 19130 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:31:47.741519 19130 rpc_server.cc:307] RPC server started. Bound to: 127.8.18.194:40993
I20260810 11:31:47.741660 19261 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.18.194:40993 every 8 connection(s)
I20260810 11:31:47.743114 19130 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786361401922362-8267-0/minicluster-data/ts-1/data/info.pb
I20260810 11:31:47.753479 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 19130
I20260810 11:31:47.753696 8267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786361401922362-8267-0/minicluster-data/ts-1/wal/instance
I20260810 11:31:47.754153 8267 external_mini_cluster.cc:1428] Setting key 49fc6d76c8aed2def774ede06478fda7
Aug 10 11:31:47 dist-test-slave-srgv krb5kdc[18885](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361507, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.194@KRBTEST.COM for kudu/127.8.18.254@KRBTEST.COM
I20260810 11:31:47.768595 19264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:47.746165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.194:44485 (local address 127.8.18.254:33599)
0810 11:31:47.746684 (+ 519us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:47.746690 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:47.747874 (+ 1184us) server_negotiation.cc:407] Connection header received
0810 11:31:47.749898 (+ 2024us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:47.749904 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:47.749983 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:47.750094 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:31:47.751825 (+ 1731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:47.752788 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:47.755330 (+ 2542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:47.755836 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:47.755947 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:47.760300 (+ 4353us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:31:47.760310 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:31:47.760317 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:31:47.760371 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:31:47.764486 (+ 4115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:31:47.765512 (+ 1026us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:31:47.765522 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:31:47.765528 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:31:47.765721 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:31:47.766284 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:31:47.766292 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:31:47.766298 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:31:47.766711 (+ 413us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:31:47.766883 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0810 11:31:47.767949 (+ 1066us) server_negotiation.cc:299] Negotiation successful
0810 11:31:47.768245 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":385,"thread_start_us":261,"threads_started":1}
I20260810 11:31:47.770180 19265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:47.746836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:33599 (local address 127.8.18.194:44485)
0810 11:31:47.747668 (+ 832us) negotiation.cc:107] Waiting for socket to connect
0810 11:31:47.747722 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0810 11:31:47.749572 (+ 1850us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:31:47.750367 (+ 795us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:31:47.750377 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:31:47.750952 (+ 575us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:31:47.751595 (+ 643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:47.751615 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:47.753557 (+ 1942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:47.753566 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:47.755165 (+ 1599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:47.755181 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:47.756133 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:47.756141 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:47.756490 (+ 349us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:47.757312 (+ 822us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:31:47.757338 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:31:47.759981 (+ 2643us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:31:47.764721 (+ 4740us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:31:47.764730 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:31:47.764745 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:31:47.765302 (+ 557us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:31:47.765894 (+ 592us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:31:47.765898 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:31:47.765901 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:31:47.766126 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:31:47.766870 (+ 744us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:31:47.766877 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:31:47.767401 (+ 524us) client_negotiation.cc:769] Sending connection context
0810 11:31:47.767790 (+ 389us) client_negotiation.cc:240] Negotiation successful
0810 11:31:47.768527 (+ 737us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":581,"thread_start_us":370,"threads_started":1}
I20260810 11:31:47.771919 19262 heartbeater.cc:347] Connected to a master server at 127.8.18.254:33599
I20260810 11:31:47.772287 19262 heartbeater.cc:464] Registering TS with master...
I20260810 11:31:47.773094 19262 heartbeater.cc:511] Master 127.8.18.254:33599 requested a full tablet report, sending...
I20260810 11:31:47.774883 18932 ts_manager.cc:194] Registered new tserver with Master: 7e7c8d1f0b0c4ac3af00f2bb1c85b15f (127.8.18.194:40993)
I20260810 11:31:47.775748 18932 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.18.194@KRBTEST.COM'} at 127.8.18.194:44485
WARNING: no policy specified for kudu/127.8.18.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.18.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.18.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.195.keytab.
Entry for principal kudu/127.8.18.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.195.keytab.
I20260810 11:31:47.821660 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786361401922362-8267-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786361401922362-8267-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786361401922362-8267-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786361401922362-8267-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.18.195:0
--local_ip_for_outbound_sockets=127.8.18.195
--webserver_interface=127.8.18.195
--webserver_port=0
--tserver_master_addrs=127.8.18.254:33599
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.195.keytab
--principal=kudu/127.8.18.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.18.212:33827
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786361401922362-8267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786361401922362-8267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786361401922362-8267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:31:48.185598 19268 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:31:48.185927 19268 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:31:48.186029 19268 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:31:48.193760 19268 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:31:48.193894 19268 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:31:48.194010 19268 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.18.195
I20260810 11:31:48.199038 19124 heartbeater.cc:503] Master 127.8.18.254:33599 was elected leader, sending a full tablet report...
I20260810 11:31:48.205241 19268 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.18.212:33827
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786361401922362-8267-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786361401922362-8267-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.195.keytab
--principal=kudu/127.8.18.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.18.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786361401922362-8267-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.18.195
--webserver_port=0
--tserver_master_addrs=127.8.18.254:33599
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.18.195
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786361401922362-8267-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:31:48.206703 19268 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:31:48.208026 19268 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:31:48.222483 19276 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:31:48.222540 19274 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:31:48.222783 19273 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:31:48.223695 19268 server_base.cc:1034] running on GCE node
I20260810 11:31:48.224330 19268 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:31:48.225369 19268 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:31:48.226660 19268 hybrid_clock.cc:648] HybridClock initialized: now 1786361508226590 us; error 60 us; skew 500 ppm
I20260810 11:31:48.226953 19268 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:31:48 dist-test-slave-srgv krb5kdc[18885](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361508, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:31:48.230360 19268 init.cc:377] Logged in from keytab as kudu/127.8.18.195@KRBTEST.COM (short username kudu)
I20260810 11:31:48.232090 19268 webserver.cc:460] Webserver started at http://127.8.18.195:45821/ using document root <none> and password file <none>
I20260810 11:31:48.232843 19268 fs_manager.cc:359] Metadata directory not provided
I20260810 11:31:48.232990 19268 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:31:48.233340 19268 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:31:48.236502 19268 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786361401922362-8267-0/minicluster-data/ts-2/data/instance:
uuid: "b6a85fa06a9646b9834dc0c10352fa01"
format_stamp: "Formatted at 2026-08-10 11:31:48 on dist-test-slave-srgv"
server_key: "916fa342ad56bbf73dfb9e5e06b28979"
server_key_iv: "20f93ad926df7534c327980ba52d3ad8"
server_key_version: "encryptionkey@0"
I20260810 11:31:48.237362 19268 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786361401922362-8267-0/minicluster-data/ts-2/wal/instance:
uuid: "b6a85fa06a9646b9834dc0c10352fa01"
format_stamp: "Formatted at 2026-08-10 11:31:48 on dist-test-slave-srgv"
server_key: "916fa342ad56bbf73dfb9e5e06b28979"
server_key_iv: "20f93ad926df7534c327980ba52d3ad8"
server_key_version: "encryptionkey@0"
I20260810 11:31:48.243184 19268 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260810 11:31:48.247583 19283 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:31:48.249138 19268 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 11:31:48.249380 19268 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786361401922362-8267-0/minicluster-data/ts-2/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786361401922362-8267-0/minicluster-data/ts-2/wal
uuid: "b6a85fa06a9646b9834dc0c10352fa01"
format_stamp: "Formatted at 2026-08-10 11:31:48 on dist-test-slave-srgv"
server_key: "916fa342ad56bbf73dfb9e5e06b28979"
server_key_iv: "20f93ad926df7534c327980ba52d3ad8"
server_key_version: "encryptionkey@0"
I20260810 11:31:48.249704 19268 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786361401922362-8267-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786361401922362-8267-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786361401922362-8267-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:31:48.281505 19268 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:31:48.286530 19268 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:31:48.286890 19268 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:31:48.288018 19268 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:31:48.289547 19268 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:31:48.289665 19268 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:31:48.289834 19268 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:31:48.289923 19268 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:31:48.328321 19268 rpc_server.cc:307] RPC server started. Bound to: 127.8.18.195:41349
I20260810 11:31:48.328377 19400 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.18.195:41349 every 8 connection(s)
I20260810 11:31:48.329947 19268 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786361401922362-8267-0/minicluster-data/ts-2/data/info.pb
I20260810 11:31:48.331465 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 19268
I20260810 11:31:48.331666 8267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786361401922362-8267-0/minicluster-data/ts-2/wal/instance
I20260810 11:31:48.332275 8267 external_mini_cluster.cc:1428] Setting key bb458968877c91dd17d1b4742c98a353
Aug 10 11:31:48 dist-test-slave-srgv krb5kdc[18885](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361508, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.195@KRBTEST.COM for kudu/127.8.18.254@KRBTEST.COM
I20260810 11:31:48.353058 19403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:48.333206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.195:50115 (local address 127.8.18.254:33599)
0810 11:31:48.333732 (+ 526us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:48.333738 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:48.335199 (+ 1461us) server_negotiation.cc:407] Connection header received
0810 11:31:48.337569 (+ 2370us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:48.337574 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:48.337660 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:48.337761 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:31:48.339545 (+ 1784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:48.340356 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:48.342182 (+ 1826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:48.342634 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:48.342759 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:48.346709 (+ 3950us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:31:48.346714 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:31:48.346718 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:31:48.346759 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:31:48.349975 (+ 3216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:31:48.350804 (+ 829us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:31:48.350809 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:31:48.350811 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:31:48.350941 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:31:48.351390 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:31:48.351394 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:31:48.351403 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:31:48.351712 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:31:48.351844 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0810 11:31:48.352572 (+ 728us) server_negotiation.cc:299] Negotiation successful
0810 11:31:48.352757 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":407,"thread_start_us":287,"threads_started":1}
I20260810 11:31:48.354249 19405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:48.333953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:33599 (local address 127.8.18.195:50115)
0810 11:31:48.334980 (+ 1027us) negotiation.cc:107] Waiting for socket to connect
0810 11:31:48.335025 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0810 11:31:48.337188 (+ 2163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:31:48.338018 (+ 830us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:31:48.338027 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:31:48.338655 (+ 628us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:31:48.339313 (+ 658us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:48.339333 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:48.340532 (+ 1199us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:48.340535 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:48.341987 (+ 1452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:48.342002 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:48.342867 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:48.342871 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:48.343091 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:48.343862 (+ 771us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:31:48.343886 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:31:48.346391 (+ 2505us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:31:48.350134 (+ 3743us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:31:48.350143 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:31:48.350159 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:31:48.350661 (+ 502us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:31:48.351038 (+ 377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:31:48.351041 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:31:48.351043 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:31:48.351268 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:31:48.351828 (+ 560us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:31:48.351835 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:31:48.352198 (+ 363us) client_negotiation.cc:769] Sending connection context
0810 11:31:48.352461 (+ 263us) client_negotiation.cc:240] Negotiation successful
0810 11:31:48.352885 (+ 424us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":720,"thread_start_us":389,"threads_started":1}
I20260810 11:31:48.355899 19401 heartbeater.cc:347] Connected to a master server at 127.8.18.254:33599
I20260810 11:31:48.356274 19401 heartbeater.cc:464] Registering TS with master...
I20260810 11:31:48.357079 19401 heartbeater.cc:511] Master 127.8.18.254:33599 requested a full tablet report, sending...
I20260810 11:31:48.358757 18932 ts_manager.cc:194] Registered new tserver with Master: b6a85fa06a9646b9834dc0c10352fa01 (127.8.18.195:41349)
I20260810 11:31:48.359653 18932 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.18.195@KRBTEST.COM'} at 127.8.18.195:50115
I20260810 11:31:48.362627 8267 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 11:31:48.382833 19403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:48.367891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45000 (local address 127.8.18.254:33599)
0810 11:31:48.368133 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:48.368138 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:48.368730 (+ 592us) server_negotiation.cc:407] Connection header received
0810 11:31:48.368934 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:48.368939 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:48.369013 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:48.369145 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:31:48.371441 (+ 2296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:48.372227 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:48.373671 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:48.374126 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:48.374234 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:48.376911 (+ 2677us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:31:48.376917 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:31:48.376920 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:31:48.376960 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:31:48.379856 (+ 2896us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:31:48.380611 (+ 755us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:31:48.380615 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:31:48.380617 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:31:48.380932 (+ 315us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:31:48.381359 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:31:48.381363 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:31:48.381365 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:31:48.381835 (+ 470us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:31:48.382030 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0810 11:31:48.382401 (+ 371us) server_negotiation.cc:299] Negotiation successful
0810 11:31:48.382593 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260810 11:31:48.387883 18932 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45000:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 11:31:48.395202 18932 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 11:31:48.427052 19417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:48.409604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.195:41349 (local address 127.0.0.1:44702)
0810 11:31:48.411071 (+ 1467us) negotiation.cc:107] Waiting for socket to connect
0810 11:31:48.411088 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 11:31:48.411199 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:31:48.413580 (+ 2381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:31:48.413592 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:31:48.413642 (+ 50us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:31:48.414271 (+ 629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:48.414292 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:48.418857 (+ 4565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:48.418867 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:48.422362 (+ 3495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:48.422397 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:48.425015 (+ 2618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:48.425035 (+ 20us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:48.425325 (+ 290us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:48.426212 (+ 887us) client_negotiation.cc:769] Sending connection context
0810 11:31:48.426363 (+ 151us) client_negotiation.cc:240] Negotiation successful
0810 11:31:48.426579 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1344,"mutex_wait_us":749,"thread_start_us":1250,"threads_started":1}
I20260810 11:31:48.427095 19418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:48.410557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44702 (local address 127.8.18.195:41349)
0810 11:31:48.412630 (+ 2073us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:48.412639 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:48.412680 (+ 41us) server_negotiation.cc:407] Connection header received
0810 11:31:48.412819 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:48.412832 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:48.413186 (+ 354us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:48.414589 (+ 1403us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:31:48.415036 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:48.418570 (+ 3534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:48.422783 (+ 4213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:48.424689 (+ 1906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:48.424840 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:48.425108 (+ 268us) server_negotiation.cc:1035] Waiting for connection context
0810 11:31:48.426502 (+ 1394us) server_negotiation.cc:299] Negotiation successful
0810 11:31:48.426720 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1834,"thread_start_us":1611,"threads_started":1}
I20260810 11:31:48.432312 19334 tablet_service.cc:1467] Processing CreateTablet for tablet d3c4b6acd0624e38ba43348107fc3cc5 (DEFAULT_TABLE table=test-table [id=604d008f16f5488da7a4bba337f0a566]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:31:48.433274 19419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:48.411414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.193:41165 (local address 127.0.0.1:48146)
0810 11:31:48.424144 (+ 12730us) negotiation.cc:107] Waiting for socket to connect
0810 11:31:48.424172 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0810 11:31:48.424503 (+ 331us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:31:48.425318 (+ 815us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:31:48.425323 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:31:48.425354 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:31:48.425993 (+ 639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:48.426008 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:48.428041 (+ 2033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:48.428050 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:48.429956 (+ 1906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:48.429980 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:48.432293 (+ 2313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:48.432301 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:48.432492 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:48.432528 (+ 36us) client_negotiation.cc:769] Sending connection context
0810 11:31:48.432633 (+ 105us) client_negotiation.cc:240] Negotiation successful
0810 11:31:48.432785 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12509,"mutex_wait_us":10,"thread_start_us":12309,"threads_started":1}
I20260810 11:31:48.433333 19420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:48.411841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48146 (local address 127.8.18.193:41165)
0810 11:31:48.423215 (+ 11374us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:48.423230 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:48.424327 (+ 1097us) server_negotiation.cc:407] Connection header received
0810 11:31:48.424735 (+ 408us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:48.424744 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:48.425062 (+ 318us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:48.425316 (+ 254us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:31:48.426247 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:48.427886 (+ 1639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:48.430187 (+ 2301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:48.431704 (+ 1517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:48.431861 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:48.432117 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0810 11:31:48.432755 (+ 638us) server_negotiation.cc:299] Negotiation successful
0810 11:31:48.432957 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11075,"thread_start_us":11406,"threads_started":1}
I20260810 11:31:48.434103 19415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:48.409173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.194:40993 (local address 127.0.0.1:48998)
0810 11:31:48.410407 (+ 1234us) negotiation.cc:107] Waiting for socket to connect
0810 11:31:48.410456 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0810 11:31:48.410644 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:31:48.425569 (+ 14925us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:31:48.425576 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:31:48.425609 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:31:48.426046 (+ 437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:48.426058 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:48.429071 (+ 3013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:48.429077 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:48.431156 (+ 2079us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:48.431169 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:48.433176 (+ 2007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:48.433182 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:48.433397 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:48.433467 (+ 70us) client_negotiation.cc:769] Sending connection context
0810 11:31:48.433545 (+ 78us) client_negotiation.cc:240] Negotiation successful
0810 11:31:48.433683 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":969,"mutex_wait_us":1,"spinlock_wait_cycles":13696,"thread_start_us":825,"threads_started":1}
I20260810 11:31:48.434360 19334 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d3c4b6acd0624e38ba43348107fc3cc5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:31:48.435298 19416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:48.409387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48998 (local address 127.8.18.194:40993)
0810 11:31:48.424932 (+ 15545us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:48.424941 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:48.424970 (+ 29us) server_negotiation.cc:407] Connection header received
0810 11:31:48.425070 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:48.425081 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:48.425330 (+ 249us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:48.425575 (+ 245us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:31:48.426286 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:48.428879 (+ 2593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:48.431351 (+ 2472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:48.432899 (+ 1548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:48.434542 (+ 1643us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:48.434727 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0810 11:31:48.434790 (+ 63us) server_negotiation.cc:299] Negotiation successful
0810 11:31:48.434929 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15369,"thread_start_us":15233,"threads_started":1}
I20260810 11:31:48.438443 19057 tablet_service.cc:1467] Processing CreateTablet for tablet d3c4b6acd0624e38ba43348107fc3cc5 (DEFAULT_TABLE table=test-table [id=604d008f16f5488da7a4bba337f0a566]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:31:48.439899 19057 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d3c4b6acd0624e38ba43348107fc3cc5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:31:48.441696 19193 tablet_service.cc:1467] Processing CreateTablet for tablet d3c4b6acd0624e38ba43348107fc3cc5 (DEFAULT_TABLE table=test-table [id=604d008f16f5488da7a4bba337f0a566]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:31:48.443897 19193 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d3c4b6acd0624e38ba43348107fc3cc5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:31:48.450542 19421 tablet_bootstrap.cc:492] T d3c4b6acd0624e38ba43348107fc3cc5 P b6a85fa06a9646b9834dc0c10352fa01: Bootstrap starting.
I20260810 11:31:48.454176 19422 tablet_bootstrap.cc:492] T d3c4b6acd0624e38ba43348107fc3cc5 P 95b8b02d46dc4bcf9efa75832f4b0cbf: Bootstrap starting.
I20260810 11:31:48.455108 19421 tablet_bootstrap.cc:654] T d3c4b6acd0624e38ba43348107fc3cc5 P b6a85fa06a9646b9834dc0c10352fa01: Neither blocks nor log segments found. Creating new log.
I20260810 11:31:48.456614 19421 log.cc:824] T d3c4b6acd0624e38ba43348107fc3cc5 P b6a85fa06a9646b9834dc0c10352fa01: Log is configured to *not* fsync() on all Append() calls
I20260810 11:31:48.458570 19422 tablet_bootstrap.cc:654] T d3c4b6acd0624e38ba43348107fc3cc5 P 95b8b02d46dc4bcf9efa75832f4b0cbf: Neither blocks nor log segments found. Creating new log.
I20260810 11:31:48.460047 19422 log.cc:824] T d3c4b6acd0624e38ba43348107fc3cc5 P 95b8b02d46dc4bcf9efa75832f4b0cbf: Log is configured to *not* fsync() on all Append() calls
I20260810 11:31:48.461900 19425 tablet_bootstrap.cc:492] T d3c4b6acd0624e38ba43348107fc3cc5 P 7e7c8d1f0b0c4ac3af00f2bb1c85b15f: Bootstrap starting.
I20260810 11:31:48.463418 19421 tablet_bootstrap.cc:492] T d3c4b6acd0624e38ba43348107fc3cc5 P b6a85fa06a9646b9834dc0c10352fa01: No bootstrap required, opened a new log
I20260810 11:31:48.463735 19421 ts_tablet_manager.cc:1397] T d3c4b6acd0624e38ba43348107fc3cc5 P b6a85fa06a9646b9834dc0c10352fa01: Time spent bootstrapping tablet: real 0.013s user 0.007s sys 0.005s
I20260810 11:31:48.464341 19422 tablet_bootstrap.cc:492] T d3c4b6acd0624e38ba43348107fc3cc5 P 95b8b02d46dc4bcf9efa75832f4b0cbf: No bootstrap required, opened a new log
I20260810 11:31:48.464627 19422 ts_tablet_manager.cc:1397] T d3c4b6acd0624e38ba43348107fc3cc5 P 95b8b02d46dc4bcf9efa75832f4b0cbf: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.004s
I20260810 11:31:48.468991 19425 tablet_bootstrap.cc:654] T d3c4b6acd0624e38ba43348107fc3cc5 P 7e7c8d1f0b0c4ac3af00f2bb1c85b15f: Neither blocks nor log segments found. Creating new log.
I20260810 11:31:48.470721 19425 log.cc:824] T d3c4b6acd0624e38ba43348107fc3cc5 P 7e7c8d1f0b0c4ac3af00f2bb1c85b15f: Log is configured to *not* fsync() on all Append() calls
I20260810 11:31:48.476205 19422 raft_consensus.cc:348] T d3c4b6acd0624e38ba43348107fc3cc5 P 95b8b02d46dc4bcf9efa75832f4b0cbf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e7c8d1f0b0c4ac3af00f2bb1c85b15f" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 40993 } } peers { permanent_uuid: "b6a85fa06a9646b9834dc0c10352fa01" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 41349 } } peers { permanent_uuid: "95b8b02d46dc4bcf9efa75832f4b0cbf" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 41165 } }
I20260810 11:31:48.476933 19422 raft_consensus.cc:374] T d3c4b6acd0624e38ba43348107fc3cc5 P 95b8b02d46dc4bcf9efa75832f4b0cbf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:31:48.477080 19422 raft_consensus.cc:729] T d3c4b6acd0624e38ba43348107fc3cc5 P 95b8b02d46dc4bcf9efa75832f4b0cbf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95b8b02d46dc4bcf9efa75832f4b0cbf, State: Initialized, Role: FOLLOWER
I20260810 11:31:48.477466 19421 raft_consensus.cc:348] T d3c4b6acd0624e38ba43348107fc3cc5 P b6a85fa06a9646b9834dc0c10352fa01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e7c8d1f0b0c4ac3af00f2bb1c85b15f" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 40993 } } peers { permanent_uuid: "b6a85fa06a9646b9834dc0c10352fa01" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 41349 } } peers { permanent_uuid: "95b8b02d46dc4bcf9efa75832f4b0cbf" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 41165 } }
I20260810 11:31:48.478062 19421 raft_consensus.cc:374] T d3c4b6acd0624e38ba43348107fc3cc5 P b6a85fa06a9646b9834dc0c10352fa01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:31:48.477934 19422 consensus_queue.cc:260] T d3c4b6acd0624e38ba43348107fc3cc5 P 95b8b02d46dc4bcf9efa75832f4b0cbf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e7c8d1f0b0c4ac3af00f2bb1c85b15f" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 40993 } } peers { permanent_uuid: "b6a85fa06a9646b9834dc0c10352fa01" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 41349 } } peers { permanent_uuid: "95b8b02d46dc4bcf9efa75832f4b0cbf" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 41165 } }
I20260810 11:31:48.478221 19421 raft_consensus.cc:729] T d3c4b6acd0624e38ba43348107fc3cc5 P b6a85fa06a9646b9834dc0c10352fa01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b6a85fa06a9646b9834dc0c10352fa01, State: Initialized, Role: FOLLOWER
I20260810 11:31:48.479003 19421 consensus_queue.cc:260] T d3c4b6acd0624e38ba43348107fc3cc5 P b6a85fa06a9646b9834dc0c10352fa01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e7c8d1f0b0c4ac3af00f2bb1c85b15f" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 40993 } } peers { permanent_uuid: "b6a85fa06a9646b9834dc0c10352fa01" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 41349 } } peers { permanent_uuid: "95b8b02d46dc4bcf9efa75832f4b0cbf" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 41165 } }
I20260810 11:31:48.481153 19425 tablet_bootstrap.cc:492] T d3c4b6acd0624e38ba43348107fc3cc5 P 7e7c8d1f0b0c4ac3af00f2bb1c85b15f: No bootstrap required, opened a new log
I20260810 11:31:48.481660 19425 ts_tablet_manager.cc:1397] T d3c4b6acd0624e38ba43348107fc3cc5 P 7e7c8d1f0b0c4ac3af00f2bb1c85b15f: Time spent bootstrapping tablet: real 0.020s user 0.012s sys 0.003s
I20260810 11:31:48.490876 19421 ts_tablet_manager.cc:1428] T d3c4b6acd0624e38ba43348107fc3cc5 P b6a85fa06a9646b9834dc0c10352fa01: Time spent starting tablet: real 0.027s user 0.018s sys 0.003s
I20260810 11:31:48.491487 19401 heartbeater.cc:503] Master 127.8.18.254:33599 was elected leader, sending a full tablet report...
I20260810 11:31:48.494136 19425 raft_consensus.cc:348] T d3c4b6acd0624e38ba43348107fc3cc5 P 7e7c8d1f0b0c4ac3af00f2bb1c85b15f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e7c8d1f0b0c4ac3af00f2bb1c85b15f" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 40993 } } peers { permanent_uuid: "b6a85fa06a9646b9834dc0c10352fa01" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 41349 } } peers { permanent_uuid: "95b8b02d46dc4bcf9efa75832f4b0cbf" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 41165 } }
I20260810 11:31:48.496304 19422 ts_tablet_manager.cc:1428] T d3c4b6acd0624e38ba43348107fc3cc5 P 95b8b02d46dc4bcf9efa75832f4b0cbf: Time spent starting tablet: real 0.032s user 0.018s sys 0.011s
I20260810 11:31:48.496563 19425 raft_consensus.cc:374] T d3c4b6acd0624e38ba43348107fc3cc5 P 7e7c8d1f0b0c4ac3af00f2bb1c85b15f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:31:48.496738 19425 raft_consensus.cc:729] T d3c4b6acd0624e38ba43348107fc3cc5 P 7e7c8d1f0b0c4ac3af00f2bb1c85b15f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e7c8d1f0b0c4ac3af00f2bb1c85b15f, State: Initialized, Role: FOLLOWER
I20260810 11:31:48.497563 19425 consensus_queue.cc:260] T d3c4b6acd0624e38ba43348107fc3cc5 P 7e7c8d1f0b0c4ac3af00f2bb1c85b15f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e7c8d1f0b0c4ac3af00f2bb1c85b15f" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 40993 } } peers { permanent_uuid: "b6a85fa06a9646b9834dc0c10352fa01" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 41349 } } peers { permanent_uuid: "95b8b02d46dc4bcf9efa75832f4b0cbf" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 41165 } }
I20260810 11:31:48.499027 19262 heartbeater.cc:503] Master 127.8.18.254:33599 was elected leader, sending a full tablet report...
I20260810 11:31:48.499840 19425 ts_tablet_manager.cc:1428] T d3c4b6acd0624e38ba43348107fc3cc5 P 7e7c8d1f0b0c4ac3af00f2bb1c85b15f: Time spent starting tablet: real 0.018s user 0.015s sys 0.000s
W20260810 11:31:48.582162 19402 tablet.cc:2369] T d3c4b6acd0624e38ba43348107fc3cc5 P b6a85fa06a9646b9834dc0c10352fa01: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 11:31:48.633359 19429 raft_consensus.cc:482] T d3c4b6acd0624e38ba43348107fc3cc5 P 7e7c8d1f0b0c4ac3af00f2bb1c85b15f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 11:31:48.634094 19429 raft_consensus.cc:504] T d3c4b6acd0624e38ba43348107fc3cc5 P 7e7c8d1f0b0c4ac3af00f2bb1c85b15f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e7c8d1f0b0c4ac3af00f2bb1c85b15f" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 40993 } } peers { permanent_uuid: "b6a85fa06a9646b9834dc0c10352fa01" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 41349 } } peers { permanent_uuid: "95b8b02d46dc4bcf9efa75832f4b0cbf" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 41165 } }
I20260810 11:31:48.635923 19429 leader_election.cc:288] T d3c4b6acd0624e38ba43348107fc3cc5 P 7e7c8d1f0b0c4ac3af00f2bb1c85b15f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b6a85fa06a9646b9834dc0c10352fa01 (127.8.18.195:41349), 95b8b02d46dc4bcf9efa75832f4b0cbf (127.8.18.193:41165)
I20260810 11:31:48.644541 19418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:48.636575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.194:40967 (local address 127.8.18.195:41349)
0810 11:31:48.636833 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:48.636840 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:48.637052 (+ 212us) server_negotiation.cc:407] Connection header received
0810 11:31:48.637324 (+ 272us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:48.637333 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:48.637469 (+ 136us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:48.637596 (+ 127us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:31:48.638333 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:48.639289 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:48.642061 (+ 2772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:48.643377 (+ 1316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:48.643586 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:48.643676 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0810 11:31:48.644126 (+ 450us) server_negotiation.cc:299] Negotiation successful
0810 11:31:48.644238 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260810 11:31:48.644773 19420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:48.637241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.194:39061 (local address 127.8.18.193:41165)
0810 11:31:48.637483 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:48.637490 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:48.638009 (+ 519us) server_negotiation.cc:407] Connection header received
0810 11:31:48.638265 (+ 256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:48.638273 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:48.638368 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:48.638489 (+ 121us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:31:48.639134 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:48.639937 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:48.642375 (+ 2438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:48.643705 (+ 1330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:48.643858 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:48.643912 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0810 11:31:48.644343 (+ 431us) server_negotiation.cc:299] Negotiation successful
0810 11:31:48.644481 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260810 11:31:48.644827 19430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:48.636268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.195:41349 (local address 127.8.18.194:40967)
0810 11:31:48.636916 (+ 648us) negotiation.cc:107] Waiting for socket to connect
0810 11:31:48.636938 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 11:31:48.637126 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:31:48.637641 (+ 515us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:31:48.637649 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:31:48.637691 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:31:48.638159 (+ 468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:48.638173 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:48.639467 (+ 1294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:48.639471 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:48.641806 (+ 2335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:48.641826 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:48.643681 (+ 1855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:48.643687 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:48.643893 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:48.643943 (+ 50us) client_negotiation.cc:769] Sending connection context
0810 11:31:48.644084 (+ 141us) client_negotiation.cc:240] Negotiation successful
0810 11:31:48.644301 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":523,"mutex_wait_us":32,"thread_start_us":345,"threads_started":1}
I20260810 11:31:48.644850 19431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:48.637076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.193:41165 (local address 127.8.18.194:39061)
0810 11:31:48.637883 (+ 807us) negotiation.cc:107] Waiting for socket to connect
0810 11:31:48.637903 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 11:31:48.638041 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:31:48.638548 (+ 507us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:31:48.638552 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:31:48.638575 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:31:48.638931 (+ 356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:48.638939 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:48.640108 (+ 1169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:48.640114 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:48.642188 (+ 2074us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:31:48.642200 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:31:48.643950 (+ 1750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:31:48.643955 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:31:48.644167 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:48.644197 (+ 30us) client_negotiation.cc:769] Sending connection context
0810 11:31:48.644285 (+ 88us) client_negotiation.cc:240] Negotiation successful
0810 11:31:48.644402 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":635,"mutex_wait_us":1,"thread_start_us":416,"threads_started":1}
I20260810 11:31:48.646209 19356 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3c4b6acd0624e38ba43348107fc3cc5" candidate_uuid: "7e7c8d1f0b0c4ac3af00f2bb1c85b15f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b6a85fa06a9646b9834dc0c10352fa01" is_pre_election: true
I20260810 11:31:48.646271 19078 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3c4b6acd0624e38ba43348107fc3cc5" candidate_uuid: "7e7c8d1f0b0c4ac3af00f2bb1c85b15f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95b8b02d46dc4bcf9efa75832f4b0cbf" is_pre_election: true
I20260810 11:31:48.646785 19356 raft_consensus.cc:2454] T d3c4b6acd0624e38ba43348107fc3cc5 P b6a85fa06a9646b9834dc0c10352fa01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e7c8d1f0b0c4ac3af00f2bb1c85b15f in term 0.
I20260810 11:31:48.646795 19078 raft_consensus.cc:2454] T d3c4b6acd0624e38ba43348107fc3cc5 P 95b8b02d46dc4bcf9efa75832f4b0cbf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e7c8d1f0b0c4ac3af00f2bb1c85b15f in term 0.
I20260810 11:31:48.647634 19149 leader_election.cc:302] T d3c4b6acd0624e38ba43348107fc3cc5 P 7e7c8d1f0b0c4ac3af00f2bb1c85b15f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e7c8d1f0b0c4ac3af00f2bb1c85b15f, b6a85fa06a9646b9834dc0c10352fa01; no voters:
I20260810 11:31:48.648159 19429 raft_consensus.cc:2781] T d3c4b6acd0624e38ba43348107fc3cc5 P 7e7c8d1f0b0c4ac3af00f2bb1c85b15f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 11:31:48.648360 19429 raft_consensus.cc:482] T d3c4b6acd0624e38ba43348107fc3cc5 P 7e7c8d1f0b0c4ac3af00f2bb1c85b15f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 11:31:48.648495 19429 raft_consensus.cc:3037] T d3c4b6acd0624e38ba43348107fc3cc5 P 7e7c8d1f0b0c4ac3af00f2bb1c85b15f [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:31:48.651006 19429 raft_consensus.cc:504] T d3c4b6acd0624e38ba43348107fc3cc5 P 7e7c8d1f0b0c4ac3af00f2bb1c85b15f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e7c8d1f0b0c4ac3af00f2bb1c85b15f" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 40993 } } peers { permanent_uuid: "b6a85fa06a9646b9834dc0c10352fa01" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 41349 } } peers { permanent_uuid: "95b8b02d46dc4bcf9efa75832f4b0cbf" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 41165 } }
I20260810 11:31:48.651667 19429 leader_election.cc:288] T d3c4b6acd0624e38ba43348107fc3cc5 P 7e7c8d1f0b0c4ac3af00f2bb1c85b15f [CANDIDATE]: Term 1 election: Requested vote from peers b6a85fa06a9646b9834dc0c10352fa01 (127.8.18.195:41349), 95b8b02d46dc4bcf9efa75832f4b0cbf (127.8.18.193:41165)
I20260810 11:31:48.652344 19356 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3c4b6acd0624e38ba43348107fc3cc5" candidate_uuid: "7e7c8d1f0b0c4ac3af00f2bb1c85b15f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b6a85fa06a9646b9834dc0c10352fa01"
I20260810 11:31:48.652424 19078 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3c4b6acd0624e38ba43348107fc3cc5" candidate_uuid: "7e7c8d1f0b0c4ac3af00f2bb1c85b15f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95b8b02d46dc4bcf9efa75832f4b0cbf"
I20260810 11:31:48.652611 19356 raft_consensus.cc:3037] T d3c4b6acd0624e38ba43348107fc3cc5 P b6a85fa06a9646b9834dc0c10352fa01 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:31:48.652661 19078 raft_consensus.cc:3037] T d3c4b6acd0624e38ba43348107fc3cc5 P 95b8b02d46dc4bcf9efa75832f4b0cbf [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:31:48.656181 19078 raft_consensus.cc:2454] T d3c4b6acd0624e38ba43348107fc3cc5 P 95b8b02d46dc4bcf9efa75832f4b0cbf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e7c8d1f0b0c4ac3af00f2bb1c85b15f in term 1.
I20260810 11:31:48.656186 19356 raft_consensus.cc:2454] T d3c4b6acd0624e38ba43348107fc3cc5 P b6a85fa06a9646b9834dc0c10352fa01 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e7c8d1f0b0c4ac3af00f2bb1c85b15f in term 1.
I20260810 11:31:48.656771 19149 leader_election.cc:302] T d3c4b6acd0624e38ba43348107fc3cc5 P 7e7c8d1f0b0c4ac3af00f2bb1c85b15f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e7c8d1f0b0c4ac3af00f2bb1c85b15f, 95b8b02d46dc4bcf9efa75832f4b0cbf; no voters:
I20260810 11:31:48.657091 19429 raft_consensus.cc:2781] T d3c4b6acd0624e38ba43348107fc3cc5 P 7e7c8d1f0b0c4ac3af00f2bb1c85b15f [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:31:48.657661 19429 raft_consensus.cc:686] T d3c4b6acd0624e38ba43348107fc3cc5 P 7e7c8d1f0b0c4ac3af00f2bb1c85b15f [term 1 LEADER]: Becoming Leader. State: Replica: 7e7c8d1f0b0c4ac3af00f2bb1c85b15f, State: Running, Role: LEADER
I20260810 11:31:48.658196 19429 consensus_queue.cc:237] T d3c4b6acd0624e38ba43348107fc3cc5 P 7e7c8d1f0b0c4ac3af00f2bb1c85b15f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e7c8d1f0b0c4ac3af00f2bb1c85b15f" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 40993 } } peers { permanent_uuid: "b6a85fa06a9646b9834dc0c10352fa01" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 41349 } } peers { permanent_uuid: "95b8b02d46dc4bcf9efa75832f4b0cbf" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 41165 } }
W20260810 11:31:48.663017 19125 tablet.cc:2369] T d3c4b6acd0624e38ba43348107fc3cc5 P 95b8b02d46dc4bcf9efa75832f4b0cbf: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 11:31:48.663518 18931 catalog_manager.cc:5495] T d3c4b6acd0624e38ba43348107fc3cc5 P 7e7c8d1f0b0c4ac3af00f2bb1c85b15f reported cstate change: term changed from 0 to 1, leader changed from <none> to 7e7c8d1f0b0c4ac3af00f2bb1c85b15f (127.8.18.194). New cstate: current_term: 1 leader_uuid: "7e7c8d1f0b0c4ac3af00f2bb1c85b15f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e7c8d1f0b0c4ac3af00f2bb1c85b15f" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 40993 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b6a85fa06a9646b9834dc0c10352fa01" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 41349 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95b8b02d46dc4bcf9efa75832f4b0cbf" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 41165 } health_report { overall_health: UNKNOWN } } }
I20260810 11:31:48.681902 19416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:48.676060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49000 (local address 127.8.18.194:40993)
0810 11:31:48.676315 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:48.676322 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:48.676439 (+ 117us) server_negotiation.cc:407] Connection header received
0810 11:31:48.676716 (+ 277us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:48.676720 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:48.676826 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:48.676995 (+ 169us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 11:31:48.677403 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:48.678172 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:48.679598 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:48.680019 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:48.680161 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:48.680711 (+ 550us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 11:31:48.681231 (+ 520us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 11:31:48.681399 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0810 11:31:48.681507 (+ 108us) server_negotiation.cc:299] Negotiation successful
0810 11:31:48.681596 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260810 11:31:48.691215 19078 raft_consensus.cc:1261] T d3c4b6acd0624e38ba43348107fc3cc5 P 95b8b02d46dc4bcf9efa75832f4b0cbf [term 1 FOLLOWER]: Refusing update from remote peer 7e7c8d1f0b0c4ac3af00f2bb1c85b15f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 11:31:48.691215 19356 raft_consensus.cc:1261] T d3c4b6acd0624e38ba43348107fc3cc5 P b6a85fa06a9646b9834dc0c10352fa01 [term 1 FOLLOWER]: Refusing update from remote peer 7e7c8d1f0b0c4ac3af00f2bb1c85b15f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 11:31:48.692477 19432 consensus_queue.cc:1035] T d3c4b6acd0624e38ba43348107fc3cc5 P 7e7c8d1f0b0c4ac3af00f2bb1c85b15f [LEADER]: Connected to new peer: Peer: permanent_uuid: "95b8b02d46dc4bcf9efa75832f4b0cbf" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 41165 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 11:31:48.692883 19429 consensus_queue.cc:1035] T d3c4b6acd0624e38ba43348107fc3cc5 P 7e7c8d1f0b0c4ac3af00f2bb1c85b15f [LEADER]: Connected to new peer: Peer: permanent_uuid: "b6a85fa06a9646b9834dc0c10352fa01" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 41349 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 11:31:48.707423 19436 mvcc.cc:205] Tried to move back new op lower bound from 7316936739586560000 to 7316936739468206080. Current Snapshot: MvccSnapshot[applied={T|T < 7316936739586560000}]
I20260810 11:31:48.723392 18931 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45000:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 11:31:48.723716 18931 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45000:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 11:31:48.728317 18931 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1cd6f5420b964bdc8502f9bb2456d0b9: Sending DeleteTablet for 3 replicas of tablet d3c4b6acd0624e38ba43348107fc3cc5
I20260810 11:31:48.729804 19193 tablet_service.cc:1514] Processing DeleteTablet for tablet d3c4b6acd0624e38ba43348107fc3cc5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:31:48 UTC) from {username='kudu', principal='kudu/127.8.18.254@KRBTEST.COM'} at 127.0.0.1:48998
I20260810 11:31:48.730525 19334 tablet_service.cc:1514] Processing DeleteTablet for tablet d3c4b6acd0624e38ba43348107fc3cc5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:31:48 UTC) from {username='kudu', principal='kudu/127.8.18.254@KRBTEST.COM'} at 127.0.0.1:44702
I20260810 11:31:48.730505 19057 tablet_service.cc:1514] Processing DeleteTablet for tablet d3c4b6acd0624e38ba43348107fc3cc5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:31:48 UTC) from {username='kudu', principal='kudu/127.8.18.254@KRBTEST.COM'} at 127.0.0.1:48146
I20260810 11:31:48.732378 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 18993
I20260810 11:31:48.756582 19447 tablet_replica.cc:330] T d3c4b6acd0624e38ba43348107fc3cc5 P 95b8b02d46dc4bcf9efa75832f4b0cbf: stopping tablet replica
I20260810 11:31:48.756896 19446 tablet_replica.cc:330] T d3c4b6acd0624e38ba43348107fc3cc5 P b6a85fa06a9646b9834dc0c10352fa01: stopping tablet replica
I20260810 11:31:48.757102 19447 raft_consensus.cc:2229] T d3c4b6acd0624e38ba43348107fc3cc5 P 95b8b02d46dc4bcf9efa75832f4b0cbf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 11:31:48.757496 19446 raft_consensus.cc:2229] T d3c4b6acd0624e38ba43348107fc3cc5 P b6a85fa06a9646b9834dc0c10352fa01 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 11:31:48.758174 19446 raft_consensus.cc:2258] T d3c4b6acd0624e38ba43348107fc3cc5 P b6a85fa06a9646b9834dc0c10352fa01 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:31:48.760842 19446 ts_tablet_manager.cc:1905] T d3c4b6acd0624e38ba43348107fc3cc5 P b6a85fa06a9646b9834dc0c10352fa01: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:31:48.761981 19445 tablet_replica.cc:330] T d3c4b6acd0624e38ba43348107fc3cc5 P 7e7c8d1f0b0c4ac3af00f2bb1c85b15f: stopping tablet replica
I20260810 11:31:48.762279 19447 raft_consensus.cc:2258] T d3c4b6acd0624e38ba43348107fc3cc5 P 95b8b02d46dc4bcf9efa75832f4b0cbf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:31:48.762693 19445 raft_consensus.cc:2229] T d3c4b6acd0624e38ba43348107fc3cc5 P 7e7c8d1f0b0c4ac3af00f2bb1c85b15f [term 1 LEADER]: Raft consensus shutting down.
I20260810 11:31:48.765870 19447 ts_tablet_manager.cc:1905] T d3c4b6acd0624e38ba43348107fc3cc5 P 95b8b02d46dc4bcf9efa75832f4b0cbf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:31:48.766186 19445 raft_consensus.cc:2258] T d3c4b6acd0624e38ba43348107fc3cc5 P 7e7c8d1f0b0c4ac3af00f2bb1c85b15f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:31:48.768851 19446 ts_tablet_manager.cc:1918] T d3c4b6acd0624e38ba43348107fc3cc5 P b6a85fa06a9646b9834dc0c10352fa01: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 11:31:48.769085 19446 log.cc:1196] T d3c4b6acd0624e38ba43348107fc3cc5 P b6a85fa06a9646b9834dc0c10352fa01: Deleting WAL directory at /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786361401922362-8267-0/minicluster-data/ts-2/wal/wals/d3c4b6acd0624e38ba43348107fc3cc5
I20260810 11:31:48.769492 19445 ts_tablet_manager.cc:1905] T d3c4b6acd0624e38ba43348107fc3cc5 P 7e7c8d1f0b0c4ac3af00f2bb1c85b15f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:31:48.769793 19446 ts_tablet_manager.cc:1939] T d3c4b6acd0624e38ba43348107fc3cc5 P b6a85fa06a9646b9834dc0c10352fa01: Deleting consensus metadata
I20260810 11:31:48.771978 18917 catalog_manager.cc:4841] TS b6a85fa06a9646b9834dc0c10352fa01 (127.8.18.195:41349): tablet d3c4b6acd0624e38ba43348107fc3cc5 (table test-table [id=604d008f16f5488da7a4bba337f0a566]) successfully deleted
I20260810 11:31:48.774400 19447 ts_tablet_manager.cc:1918] T d3c4b6acd0624e38ba43348107fc3cc5 P 95b8b02d46dc4bcf9efa75832f4b0cbf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 11:31:48.774717 19447 log.cc:1196] T d3c4b6acd0624e38ba43348107fc3cc5 P 95b8b02d46dc4bcf9efa75832f4b0cbf: Deleting WAL directory at /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786361401922362-8267-0/minicluster-data/ts-0/wal/wals/d3c4b6acd0624e38ba43348107fc3cc5
I20260810 11:31:48.775380 19447 ts_tablet_manager.cc:1939] T d3c4b6acd0624e38ba43348107fc3cc5 P 95b8b02d46dc4bcf9efa75832f4b0cbf: Deleting consensus metadata
Aug 10 11:31:48 dist-test-slave-srgv krb5kdc[18885](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361505, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.18.193@KRBTEST.COM
I20260810 11:31:48.777578 18917 catalog_manager.cc:4841] TS 95b8b02d46dc4bcf9efa75832f4b0cbf (127.8.18.193:41165): tablet d3c4b6acd0624e38ba43348107fc3cc5 (table test-table [id=604d008f16f5488da7a4bba337f0a566]) successfully deleted
I20260810 11:31:48.780647 19445 ts_tablet_manager.cc:1918] T d3c4b6acd0624e38ba43348107fc3cc5 P 7e7c8d1f0b0c4ac3af00f2bb1c85b15f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 11:31:48.780942 19445 log.cc:1196] T d3c4b6acd0624e38ba43348107fc3cc5 P 7e7c8d1f0b0c4ac3af00f2bb1c85b15f: Deleting WAL directory at /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786361401922362-8267-0/minicluster-data/ts-1/wal/wals/d3c4b6acd0624e38ba43348107fc3cc5
I20260810 11:31:48.781668 19445 ts_tablet_manager.cc:1939] T d3c4b6acd0624e38ba43348107fc3cc5 P 7e7c8d1f0b0c4ac3af00f2bb1c85b15f: Deleting consensus metadata
I20260810 11:31:48.783795 18914 catalog_manager.cc:4841] TS 7e7c8d1f0b0c4ac3af00f2bb1c85b15f (127.8.18.194:40993): tablet d3c4b6acd0624e38ba43348107fc3cc5 (table test-table [id=604d008f16f5488da7a4bba337f0a566]) successfully deleted
I20260810 11:31:48.786672 19420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:48.734408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48152 (local address 127.8.18.193:41165)
0810 11:31:48.734690 (+ 282us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:48.734700 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:48.748633 (+ 13933us) server_negotiation.cc:407] Connection header received
0810 11:31:48.753882 (+ 5249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:48.753892 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:48.754024 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:48.754124 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:31:48.757473 (+ 3349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:48.759109 (+ 1636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:48.762886 (+ 3777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:48.763582 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:48.763695 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:48.778666 (+ 14971us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:31:48.778684 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:31:48.778704 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:31:48.778762 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:31:48.783155 (+ 4393us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:31:48.784335 (+ 1180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:31:48.784351 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:31:48.784354 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:31:48.784507 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:31:48.784949 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:31:48.784954 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:31:48.784957 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:31:48.785341 (+ 384us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:31:48.785693 (+ 352us) server_negotiation.cc:1035] Waiting for connection context
0810 11:31:48.786032 (+ 339us) server_negotiation.cc:299] Negotiation successful
0810 11:31:48.786287 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260810 11:31:48.787433 19119 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:31:48.881681 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 18993
I20260810 11:31:48.908324 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 19130
Aug 10 11:31:48 dist-test-slave-srgv krb5kdc[18885](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361505, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.18.194@KRBTEST.COM
I20260810 11:31:48.928798 19416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:48.909274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49010 (local address 127.8.18.194:40993)
0810 11:31:48.909512 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:48.909517 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:48.909581 (+ 64us) server_negotiation.cc:407] Connection header received
0810 11:31:48.909854 (+ 273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:48.909858 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:48.909949 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:48.910045 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:31:48.912079 (+ 2034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:48.913053 (+ 974us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:48.915068 (+ 2015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:48.915779 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:48.915913 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:48.921552 (+ 5639us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:31:48.921571 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:31:48.921590 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:31:48.921664 (+ 74us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:31:48.925242 (+ 3578us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:31:48.926273 (+ 1031us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:31:48.926286 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:31:48.926288 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:31:48.926471 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:31:48.926976 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:31:48.926980 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:31:48.926983 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:31:48.927395 (+ 412us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:31:48.927789 (+ 394us) server_negotiation.cc:1035] Waiting for connection context
0810 11:31:48.928143 (+ 354us) server_negotiation.cc:299] Negotiation successful
0810 11:31:48.928398 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260810 11:31:48.929582 19257 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:31:49.047813 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 19130
I20260810 11:31:49.069025 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 19268
Aug 10 11:31:49 dist-test-slave-srgv krb5kdc[18885](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361505, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.18.195@KRBTEST.COM
I20260810 11:31:49.087023 19418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:31:49.070035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44710 (local address 127.8.18.195:41349)
0810 11:31:49.070251 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0810 11:31:49.070256 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:31:49.070277 (+ 21us) server_negotiation.cc:407] Connection header received
0810 11:31:49.070370 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:31:49.070374 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:31:49.070439 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:31:49.070544 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:31:49.072078 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:49.072851 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:49.074450 (+ 1599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:31:49.074899 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:31:49.075010 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:31:49.080448 (+ 5438us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:31:49.080467 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:31:49.080483 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:31:49.080527 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:31:49.083806 (+ 3279us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:31:49.084751 (+ 945us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:31:49.084760 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:31:49.084762 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:31:49.084927 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:31:49.085395 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:31:49.085400 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:31:49.085402 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:31:49.085934 (+ 532us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:31:49.086197 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0810 11:31:49.086525 (+ 328us) server_negotiation.cc:299] Negotiation successful
0810 11:31:49.086738 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260810 11:31:49.087719 19396 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:31:49.180866 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 19268
I20260810 11:31:49.199455 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 18898
I20260810 11:31:49.200810 18962 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:31:49.315774 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 18898
2026-08-10T11:31:49Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (4787 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786361401922362-8267-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 11:31:49 dist-test-slave-srgv krb5kdc[19455](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 11:31:49 dist-test-slave-srgv krb5kdc[19455](info): set up 2 sockets
Aug 10 11:31:49 dist-test-slave-srgv krb5kdc[19455](info): commencing operation
krb5kdc: starting...
W20260810 11:31:50.557677 8267 mini_kdc.cc:121] Time spent starting KDC: real 1.193s user 0.010s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786361401922362-8267-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786361401922362-8267-0/krb5kdc/test-user.keytab.
Aug 10 11:31:50 dist-test-slave-srgv krb5kdc[19455](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361510, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T11:31:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T11:31:50Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.8.18.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.8.18.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.8.18.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786361401922362-8267-0/krb5kdc/rangeradmin_127.8.18.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.8.18.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786361401922362-8267-0/krb5kdc/rangeradmin_127.8.18.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.8.18.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.8.18.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.8.18.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786361401922362-8267-0/krb5kdc/rangerlookup_127.8.18.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.8.18.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786361401922362-8267-0/krb5kdc/rangerlookup_127.8.18.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.8.18.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.8.18.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.8.18.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786361401922362-8267-0/krb5kdc/HTTP_127.8.18.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.8.18.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786361401922362-8267-0/krb5kdc/HTTP_127.8.18.212@KRBTEST.COM.keytab.
I20260810 11:31:50.977437 8267 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786361401922362-8267-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-task_r3aKW/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786361401922362-8267-0/minicluster-data/postgres -l logfile start
2026-08-10 11:31:53.937 UTC [19485] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-10 11:31:53.937 UTC [19485] LOG: listening on IPv4 address "127.8.18.212", port 57597
2026-08-10 11:31:53.941 UTC [19485] LOG: listening on Unix socket "/tmp/.s.PGSQL.57597"
2026-08-10 11:31:53.967 UTC [19488] LOG: database system was shut down at 2026-08-10 11:31:52 UTC
2026-08-10 11:31:53.971 UTC [19485] LOG: database system is ready to accept connections
I20260810 11:31:54.993384 8267 mini_postgres.cc:96] Postgres bound to 57597
2026-08-10 11:31:55.017 UTC [19496] FATAL: database "slave" does not exist
127.8.18.212:57597 - accepting connections
I20260810 11:31:55.018846 8267 mini_ranger.cc:161] Starting Ranger...
I20260810 11:31:55.053313 8267 mini_ranger.cc:84] Created miniranger Postgres user
I20260810 11:31:55.635907 8267 mini_ranger.cc:87] Created ranger Postgres database
I20260810 11:31:55.636132 8267 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786361401922362-8267-0/minicluster-data/ranger-admin
2026-08-10 11:31:55,992 [I] DB FLAVOR :POSTGRES
2026-08-10 11:31:55,992 [I] --------- Verifying Ranger DB connection ---------
2026-08-10 11:31:55,992 [I] Checking connection..
2026-08-10 11:31:55,992 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_r3aKW/build/asan/bin/postgresql.jar:/tmp/dist-test-task_r3aKW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.18.212:57597/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-10 11:31:56,261 [I] Checking connection passed.
2026-08-10 11:31:56,262 [I] --------- Verifying version history table ---------
2026-08-10 11:31:56,262 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_r3aKW/build/asan/bin/postgresql.jar:/tmp/dist-test-task_r3aKW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.18.212:57597/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-10 11:31:56,551 [I] Table x_db_version_h does not exist in database ranger
2026-08-10 11:31:56,552 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_r3aKW/build/asan/bin/postgresql.jar:/tmp/dist-test-task_r3aKW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.18.212:57597/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-10 11:31:56,812 [I] Table x_db_version_h does not exist in database ranger
2026-08-10 11:31:56,812 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-10 11:31:56,813 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_r3aKW/build/asan/bin/postgresql.jar:/tmp/dist-test-task_r3aKW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.18.212:57597/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task_r3aKW/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-10 11:31:57.094 UTC [19561] WARNING: there is no transaction in progress
2026-08-10 11:31:57,101 [I] create_dbversion_catalog.sql file imported successfully
2026-08-10 11:31:57,102 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_r3aKW/build/asan/bin/postgresql.jar:/tmp/dist-test-task_r3aKW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.18.212:57597/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-10 11:31:57,385 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-10 11:31:57,385 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-10 11:31:57,385 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_r3aKW/build/asan/bin/postgresql.jar:/tmp/dist-test-task_r3aKW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.18.212:57597/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-10 11:31:57,638 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_r3aKW/build/asan/bin/postgresql.jar:/tmp/dist-test-task_r3aKW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.18.212:57597/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-10 11:31:57,908 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_r3aKW/build/asan/bin/postgresql.jar:/tmp/dist-test-task_r3aKW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.18.212:57597/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-srgv.c.gcp-upstream.internal','N') ;"
2026-08-10 11:31:58,153 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_r3aKW/build/asan/bin/postgresql.jar:/tmp/dist-test-task_r3aKW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.18.212:57597/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-10 11:31:58,427 [I] Table x_portal_user does not exist in database ranger
2026-08-10 11:31:58,427 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_r3aKW/build/asan/bin/postgresql.jar:/tmp/dist-test-task_r3aKW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.18.212:57597/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-10 11:31:58,715 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-10 11:31:58,715 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_r3aKW/build/asan/bin/postgresql.jar:/tmp/dist-test-task_r3aKW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.18.212:57597/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-10 11:31:59,001 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_r3aKW/build/asan/bin/postgresql.jar:/tmp/dist-test-task_r3aKW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.18.212:57597/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-10 11:31:59,265 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-10 11:31:59,265 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_r3aKW/build/asan/bin/postgresql.jar:/tmp/dist-test-task_r3aKW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.18.212:57597/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task_r3aKW/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-10 11:32:00.371 UTC [19672] WARNING: there is no transaction in progress
2026-08-10 11:32:00.391 UTC [19672] WARNING: there is no transaction in progress
2026-08-10 11:32:00.410 UTC [19672] WARNING: there is no transaction in progress
2026-08-10 11:32:00.429 UTC [19672] WARNING: there is no transaction in progress
2026-08-10 11:32:00.448 UTC [19672] WARNING: there is no transaction in progress
2026-08-10 11:32:00.468 UTC [19672] WARNING: there is no transaction in progress
2026-08-10 11:32:00.531 UTC [19672] WARNING: there is no transaction in progress
2026-08-10 11:32:00.540 UTC [19672] WARNING: there is no transaction in progress
2026-08-10 11:32:00.557 UTC [19672] WARNING: there is no transaction in progress
2026-08-10 11:32:00.570 UTC [19672] WARNING: there is no transaction in progress
2026-08-10 11:32:00.583 UTC [19672] WARNING: there is no transaction in progress
2026-08-10 11:32:00.598 UTC [19672] WARNING: there is no transaction in progress
2026-08-10 11:32:00.609 UTC [19672] WARNING: there is no transaction in progress
2026-08-10 11:32:00.618 UTC [19672] WARNING: there is no transaction in progress
2026-08-10 11:32:00.626 UTC [19672] WARNING: there is no transaction in progress
2026-08-10 11:32:00.635 UTC [19672] WARNING: there is no transaction in progress
2026-08-10 11:32:01,459 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-10 11:32:01,460 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_r3aKW/build/asan/bin/postgresql.jar:/tmp/dist-test-task_r3aKW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.18.212:57597/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-10 11:32:01,742 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-10 11:32:01,742 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_r3aKW/build/asan/bin/postgresql.jar:/tmp/dist-test-task_r3aKW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.18.212:57597/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-10 11:32:01,997 [I] Table x_portal_user already exists in database 'ranger'
2026-08-10 11:32:01,998 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_r3aKW/build/asan/bin/postgresql.jar:/tmp/dist-test-task_r3aKW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.18.212:57597/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-10 11:32:02,253 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-10 11:32:02,253 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_r3aKW/build/asan/bin/postgresql.jar:/tmp/dist-test-task_r3aKW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.18.212:57597/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-10 11:32:02,519 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_r3aKW/build/asan/bin/postgresql.jar:/tmp/dist-test-task_r3aKW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.18.212:57597/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-10 11:32:02,800 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_r3aKW/build/asan/bin/postgresql.jar:/tmp/dist-test-task_r3aKW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.18.212:57597/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-srgv.c.gcp-upstream.internal';"
2026-08-10 11:32:03,059 [I] CORE_DB_SCHEMA import status has been updated
2026-08-10 11:32:03,059 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_r3aKW/build/asan/bin/postgresql.jar:/tmp/dist-test-task_r3aKW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.18.212:57597/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-10 11:32:03,324 [I] DB_PATCHES have already been applied
I20260810 11:32:03.331212 8267 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786361401922362-8267-0/minicluster-data/ranger-admin:/tmp/dist-test-task_r3aKW/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-task_r3aKW/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-task_r3aKW/build/asan/bin/postgresql.jar:/tmp/dist-test-task_r3aKW/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260810 11:32:03.331386 8267 mini_ranger.cc:193] Using host: 127.8.18.212
I20260810 11:32:03.351397 8267 mini_ranger.cc:227] Ranger admin URL: http://127.8.18.212:32811
Aug 10, 2026 11:32:03 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 10, 2026 11:32:03 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 10, 2026 11:32:03 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786361401922362-8267-0/minicluster-data/ranger-admin/ews/webapp
Aug 10, 2026 11:32:03 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786361401922362-8267-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 10, 2026 11:32:03 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786361401922362-8267-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 10, 2026 11:32:04 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786361401922362-8267-0/minicluster-data/ranger-admin/ews/webapp].
Aug 10, 2026 11:32:04 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260810 11:32:04.538514 8267 mini_ranger.cc:160] Time spent starting Ranger: real 9.520s user 0.000s sys 0.062s
Aug 10, 2026 11:32:25 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 10, 2026 11:32:25 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 10, 2026 11:32:25 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 10, 2026 11:32:25 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 10, 2026 11:32:25 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 10, 2026 11:32:25 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 10, 2026 11:32:25 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 10, 2026 11:32:25 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 10, 2026 11:32:25 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 10, 2026 11:32:25 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 10, 2026 11:32:25 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 10, 2026 11:32:25 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 10, 2026 11:32:25 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 10, 2026 11:32:25 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 10, 2026 11:32:25 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 10, 2026 11:32:25 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 10, 2026 11:32:25 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 10, 2026 11:32:26 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260810 11:32:27.532763 8267 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.8.18.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.8.18.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.8.18.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786361401922362-8267-0/krb5kdc/rangerkms_127.8.18.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.8.18.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786361401922362-8267-0/krb5kdc/rangerkms_127.8.18.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.8.18.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.8.18.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.8.18.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786361401922362-8267-0/krb5kdc/HTTP_127.8.18.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.8.18.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786361401922362-8267-0/krb5kdc/HTTP_127.8.18.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 10 11:32:27 dist-test-slave-srgv krb5kdc[19455](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361547, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260810 11:32:27.772845 8267 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260810 11:32:27.800504 8267 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260810 11:32:28.844683 8267 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260810 11:32:28.844872 8267 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786361401922362-8267-0/minicluster-data/ranger-kms
I20260810 11:32:28.845046 8267 mini_ranger_kms.cc:227] Using postgres at 127.8.18.212:57597
2026-08-10 11:32:29,452 [I] DB FLAVOR :POSTGRES
2026-08-10 11:32:29,452 [I] --------- Verifying Ranger DB connection ---------
2026-08-10 11:32:29,452 [I] Checking connection
2026-08-10 11:32:29,452 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_r3aKW/build/asan/bin/postgresql.jar:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786361401922362-8267-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.18.212:57597/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-10 11:32:29,710 [I] connection success
2026-08-10 11:32:29,710 [I] --------- Verifying Ranger DB tables ---------
2026-08-10 11:32:29,710 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-10 11:32:29,710 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_r3aKW/build/asan/bin/postgresql.jar:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786361401922362-8267-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.18.212:57597/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-10 11:32:29,985 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-10 11:32:29,985 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-10 11:32:29,986 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-10 11:32:29,986 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_r3aKW/build/asan/bin/postgresql.jar:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786361401922362-8267-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.18.212:57597/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786361401922362-8267-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-10 11:32:30,298 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260810 11:32:30.834662 8267 mini_ranger_kms.cc:310] Created kms service
I20260810 11:32:31.072113 8267 mini_ranger_kms.cc:326] Created kudu user
I20260810 11:32:31.124109 8267 mini_ranger_kms.cc:343] Created rangerkms user
Aug 10 11:32:31 dist-test-slave-srgv krb5kdc[19455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361547, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.8.18.212@KRBTEST.COM
I20260810 11:32:31.616047 8267 mini_ranger_kms.cc:384] Added ranger policy
I20260810 11:32:31.617695 8267 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786361401922362-8267-0/minicluster-data/ranger-kms:/tmp/dist-test-task_r3aKW/build/asan/bin/postgresql.jar:/tmp/dist-test-task_r3aKW/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-task_r3aKW/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260810 11:32:31.617774 8267 mini_ranger_kms.cc:240] Using host: 127.8.18.212
I20260810 11:32:31.647177 8267 mini_ranger_kms.cc:276] Ranger KMS PID: 19856
I20260810 11:32:31.647415 8267 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.8.18.212:36593
Aug 10, 2026 11:32:32 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 10, 2026 11:32:32 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 10, 2026 11:32:32 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786361401922362-8267-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 10, 2026 11:32:32 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786361401922362-8267-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 10, 2026 11:32:33 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786361401922362-8267-0/minicluster-data/ranger-kms/ews/webapp].
Aug 10, 2026 11:32:33 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260810 11:32:34.017565 8267 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.245s user 0.149s sys 0.485s
I20260810 11:32:34.017966 8267 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260810 11:32:34.018263 8267 mini_ranger_kms.cc:401] 127.8.18.212:36593/kms/v1/keys
Aug 10 11:32:34 dist-test-slave-srgv krb5kdc[19455](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786361554, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.8.18.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 10 11:32:34 dist-test-slave-srgv krb5kdc[19455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361554, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.8.18.212@KRBTEST.COM for HTTP/127.8.18.212@KRBTEST.COM
Aug 10 11:32:35 dist-test-slave-srgv krb5kdc[19455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361554, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.8.18.212@KRBTEST.COM for HTTP/127.8.18.212@KRBTEST.COM
Aug 10 11:32:37 dist-test-slave-srgv krb5kdc[19455](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361557, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.8.18.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.18.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.18.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.254.keytab.
Entry for principal kudu/127.8.18.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.254.keytab.
I20260810 11:32:37.618203 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786361401922362-8267-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786361401922362-8267-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786361401922362-8267-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.18.254:45021
--webserver_interface=127.8.18.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.254.keytab
--principal=kudu/127.8.18.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.18.212:43951
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.18.254:45021
--ranger_config_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786361401922362-8267-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.18.212:36593/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786361401922362-8267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786361401922362-8267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786361401922362-8267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:32:38.258463 19907 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:32:38.258917 19907 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:32:38.259120 19907 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:32:38.272626 19907 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 11:32:38.273027 19907 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:32:38.273228 19907 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:32:38.273444 19907 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 11:32:38.273584 19907 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 11:32:38.289366 19907 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.18.212:43951
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786361401922362-8267-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786361401922362-8267-0/minicluster-data/master-0/wal
--ranger_kms_url=127.8.18.212:36593/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.8.18.254:45021
--ranger_config_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786361401922362-8267-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.254.keytab
--principal=kudu/127.8.18.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.18.254:45021
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.18.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786361401922362-8267-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:32:38.291127 19907 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:32:38.292990 19907 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:32:38.313648 19915 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:32:38.313741 19913 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:32:38.313928 19912 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:32:38.316018 19907 server_base.cc:1034] running on GCE node
I20260810 11:32:38.317201 19907 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:32:38.319391 19907 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:32:38.320693 19907 hybrid_clock.cc:648] HybridClock initialized: now 1786361558320633 us; error 60 us; skew 500 ppm
I20260810 11:32:38.321250 19907 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:32:38 dist-test-slave-srgv krb5kdc[19455](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361558, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:32:38.326612 19907 init.cc:377] Logged in from keytab as kudu/127.8.18.254@KRBTEST.COM (short username kudu)
I20260810 11:32:38.329160 19907 webserver.cc:460] Webserver started at http://127.8.18.254:37281/ using document root <none> and password file <none>
I20260810 11:32:38.330298 19907 fs_manager.cc:359] Metadata directory not provided
I20260810 11:32:38.330600 19907 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:32:38.331054 19907 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 11:32:38 dist-test-slave-srgv krb5kdc[19455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361558, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.254@KRBTEST.COM for HTTP/127.8.18.212@KRBTEST.COM
I20260810 11:32:38.403432 19907 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786361401922362-8267-0/minicluster-data/master-0/data/instance:
uuid: "02097e106d234d4781a5e662f93ba496"
format_stamp: "Formatted at 2026-08-10 11:32:38 on dist-test-slave-srgv"
server_key: "7a0cf716cea1fbdc036ca60e096bf49e"
server_key_iv: "334b4e8ead65436efb6e4bf52beafac0"
server_key_version: "kuduclusterkey@0"
I20260810 11:32:38.404706 19907 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786361401922362-8267-0/minicluster-data/master-0/wal/instance:
uuid: "02097e106d234d4781a5e662f93ba496"
format_stamp: "Formatted at 2026-08-10 11:32:38 on dist-test-slave-srgv"
server_key: "7a0cf716cea1fbdc036ca60e096bf49e"
server_key_iv: "334b4e8ead65436efb6e4bf52beafac0"
server_key_version: "kuduclusterkey@0"
I20260810 11:32:38.413064 19907 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.009s sys 0.001s
I20260810 11:32:38.437914 19922 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:32:38.439898 19907 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.002s
I20260810 11:32:38.440224 19907 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786361401922362-8267-0/minicluster-data/master-0/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786361401922362-8267-0/minicluster-data/master-0/wal
uuid: "02097e106d234d4781a5e662f93ba496"
format_stamp: "Formatted at 2026-08-10 11:32:38 on dist-test-slave-srgv"
server_key: "7a0cf716cea1fbdc036ca60e096bf49e"
server_key_iv: "334b4e8ead65436efb6e4bf52beafac0"
server_key_version: "kuduclusterkey@0"
I20260810 11:32:38.440624 19907 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786361401922362-8267-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786361401922362-8267-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786361401922362-8267-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:32:38.473923 19907 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:32:38.480726 19907 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:32:38.481349 19907 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:32:38.529588 19907 rpc_server.cc:307] RPC server started. Bound to: 127.8.18.254:45021
I20260810 11:32:38.529789 19975 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.18.254:45021 every 8 connection(s)
I20260810 11:32:38.531708 19907 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
I20260810 11:32:38.536368 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 19907
I20260810 11:32:38.536615 8267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786361401922362-8267-0/minicluster-data/master-0/wal/instance
I20260810 11:32:38.537623 8267 external_mini_cluster.cc:1428] Setting key 5026dd3ce48bd1f629468c242341deb4
I20260810 11:32:38.538614 19976 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 10 11:32:38 dist-test-slave-srgv krb5kdc[19455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361557, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.18.254@KRBTEST.COM
I20260810 11:32:38.563853 19976 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 02097e106d234d4781a5e662f93ba496: Bootstrap starting.
I20260810 11:32:38.569535 19976 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 02097e106d234d4781a5e662f93ba496: Neither blocks nor log segments found. Creating new log.
I20260810 11:32:38.571986 19976 log.cc:824] T 00000000000000000000000000000000 P 02097e106d234d4781a5e662f93ba496: Log is configured to *not* fsync() on all Append() calls
I20260810 11:32:38.574846 19980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:32:38.540705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43460 (local address 127.8.18.254:45021)
0810 11:32:38.542894 (+ 2189us) server_negotiation.cc:206] Beginning negotiation
0810 11:32:38.542911 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0810 11:32:38.543187 (+ 276us) server_negotiation.cc:407] Connection header received
0810 11:32:38.547743 (+ 4556us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:32:38.547779 (+ 36us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:32:38.548344 (+ 565us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:32:38.548760 (+ 416us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:32:38.550998 (+ 2238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:32:38.553180 (+ 2182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:32:38.555530 (+ 2350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:32:38.556508 (+ 978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:32:38.556689 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:32:38.562967 (+ 6278us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:32:38.563013 (+ 46us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:32:38.563039 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:32:38.563105 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:32:38.567991 (+ 4886us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:32:38.569198 (+ 1207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:32:38.569206 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:32:38.569214 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:32:38.569580 (+ 366us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:32:38.570326 (+ 746us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:32:38.570330 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:32:38.570332 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:32:38.571041 (+ 709us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:32:38.571403 (+ 362us) server_negotiation.cc:1035] Waiting for connection context
0810 11:32:38.571899 (+ 496us) server_negotiation.cc:299] Negotiation successful
0810 11:32:38.572440 (+ 541us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1779,"thread_start_us":1416,"threads_started":1}
I20260810 11:32:38.577625 19976 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 02097e106d234d4781a5e662f93ba496: No bootstrap required, opened a new log
I20260810 11:32:38.592645 19976 raft_consensus.cc:348] T 00000000000000000000000000000000 P 02097e106d234d4781a5e662f93ba496 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02097e106d234d4781a5e662f93ba496" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 45021 } }
I20260810 11:32:38.593739 19976 raft_consensus.cc:374] T 00000000000000000000000000000000 P 02097e106d234d4781a5e662f93ba496 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:32:38.594026 19976 raft_consensus.cc:729] T 00000000000000000000000000000000 P 02097e106d234d4781a5e662f93ba496 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02097e106d234d4781a5e662f93ba496, State: Initialized, Role: FOLLOWER
I20260810 11:32:38.595235 19976 consensus_queue.cc:260] T 00000000000000000000000000000000 P 02097e106d234d4781a5e662f93ba496 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02097e106d234d4781a5e662f93ba496" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 45021 } }
I20260810 11:32:38.596065 19976 raft_consensus.cc:388] T 00000000000000000000000000000000 P 02097e106d234d4781a5e662f93ba496 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 11:32:38.596496 19976 raft_consensus.cc:482] T 00000000000000000000000000000000 P 02097e106d234d4781a5e662f93ba496 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 11:32:38.596827 19976 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 02097e106d234d4781a5e662f93ba496 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:32:38.600654 19976 raft_consensus.cc:504] T 00000000000000000000000000000000 P 02097e106d234d4781a5e662f93ba496 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02097e106d234d4781a5e662f93ba496" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 45021 } }
I20260810 11:32:38.601516 19976 leader_election.cc:302] T 00000000000000000000000000000000 P 02097e106d234d4781a5e662f93ba496 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 02097e106d234d4781a5e662f93ba496; no voters:
I20260810 11:32:38.602669 19976 leader_election.cc:288] T 00000000000000000000000000000000 P 02097e106d234d4781a5e662f93ba496 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 11:32:38.602873 19984 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 02097e106d234d4781a5e662f93ba496 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:32:38.605842 19984 raft_consensus.cc:686] T 00000000000000000000000000000000 P 02097e106d234d4781a5e662f93ba496 [term 1 LEADER]: Becoming Leader. State: Replica: 02097e106d234d4781a5e662f93ba496, State: Running, Role: LEADER
I20260810 11:32:38.607025 19976 sys_catalog.cc:564] T 00000000000000000000000000000000 P 02097e106d234d4781a5e662f93ba496 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 11:32:38.606918 19984 consensus_queue.cc:237] T 00000000000000000000000000000000 P 02097e106d234d4781a5e662f93ba496 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02097e106d234d4781a5e662f93ba496" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 45021 } }
I20260810 11:32:38.612313 19986 sys_catalog.cc:455] T 00000000000000000000000000000000 P 02097e106d234d4781a5e662f93ba496 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 02097e106d234d4781a5e662f93ba496. Latest consensus state: current_term: 1 leader_uuid: "02097e106d234d4781a5e662f93ba496" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02097e106d234d4781a5e662f93ba496" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 45021 } } }
I20260810 11:32:38.612197 19985 sys_catalog.cc:455] T 00000000000000000000000000000000 P 02097e106d234d4781a5e662f93ba496 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "02097e106d234d4781a5e662f93ba496" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02097e106d234d4781a5e662f93ba496" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 45021 } } }
I20260810 11:32:38.613235 19986 sys_catalog.cc:458] T 00000000000000000000000000000000 P 02097e106d234d4781a5e662f93ba496 [sys.catalog]: This master's current role is: LEADER
I20260810 11:32:38.613348 19985 sys_catalog.cc:458] T 00000000000000000000000000000000 P 02097e106d234d4781a5e662f93ba496 [sys.catalog]: This master's current role is: LEADER
I20260810 11:32:38.621284 19992 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 11:32:38.631206 19992 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 11:32:38.632200 19976 ranger_client.cc:318] Using new properties file: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786361401922362-8267-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260810 11:32:38.669806 19992 catalog_manager.cc:1347] Generated new cluster ID: 535fcea55646462fbf21d76df36727ed
I20260810 11:32:38.670085 19992 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 11:32:38.713775 19992 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 11:32:38.715598 19992 catalog_manager.cc:1504] Loading token signing keys...
I20260810 11:32:38.728711 19992 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 02097e106d234d4781a5e662f93ba496: Generated new TSK 0
I20260810 11:32:38.730101 19992 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.18.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.18.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.18.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.193.keytab.
Entry for principal kudu/127.8.18.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.193.keytab.
I20260810 11:32:39.703647 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786361401922362-8267-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786361401922362-8267-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786361401922362-8267-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786361401922362-8267-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.18.212:36593/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.18.193:0
--local_ip_for_outbound_sockets=127.8.18.193
--webserver_interface=127.8.18.193
--webserver_port=0
--tserver_master_addrs=127.8.18.254:45021
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.193.keytab
--principal=kudu/127.8.18.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.18.212:43951
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786361401922362-8267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786361401922362-8267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786361401922362-8267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 11:32:40 dist-test-slave-srgv krb5kdc[19455](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786361560, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260810 11:32:40.430646 20025 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:32:40.431349 20025 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:32:40.431708 20025 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:32:40.451371 20025 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:32:40.451726 20025 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:32:40.452015 20025 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.18.193
I20260810 11:32:40.477685 20025 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.18.212:43951
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786361401922362-8267-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786361401922362-8267-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.8.18.212:36593/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.193.keytab
--principal=kudu/127.8.18.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.18.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786361401922362-8267-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.18.193
--webserver_port=0
--tserver_master_addrs=127.8.18.254:45021
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.18.193
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786361401922362-8267-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:32:40.479996 20025 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:32:40.482476 20025 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:32:40.515550 20035 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:32:40.515816 20036 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:32:40.517604 20038 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:32:40.524369 20025 server_base.cc:1034] running on GCE node
I20260810 11:32:40.525341 20025 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:32:40.527587 20025 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:32:40.529841 20025 hybrid_clock.cc:648] HybridClock initialized: now 1786361560529635 us; error 145 us; skew 500 ppm
I20260810 11:32:40.530457 20025 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:32:40 dist-test-slave-srgv krb5kdc[19455](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361560, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:32:40.536451 20025 init.cc:377] Logged in from keytab as kudu/127.8.18.193@KRBTEST.COM (short username kudu)
I20260810 11:32:40.540540 20025 webserver.cc:460] Webserver started at http://127.8.18.193:33487/ using document root <none> and password file <none>
I20260810 11:32:40.541736 20025 fs_manager.cc:359] Metadata directory not provided
I20260810 11:32:40.542045 20025 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:32:40.542593 20025 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 11:32:40 dist-test-slave-srgv krb5kdc[19455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361560, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.193@KRBTEST.COM for HTTP/127.8.18.212@KRBTEST.COM
I20260810 11:32:40.575438 20025 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786361401922362-8267-0/minicluster-data/ts-0/data/instance:
uuid: "130ff4307aa34e638e62a7e068c3595c"
format_stamp: "Formatted at 2026-08-10 11:32:40 on dist-test-slave-srgv"
server_key: "a531a836a9e5b9b03f5b30f524636c6a"
server_key_iv: "1a8d81596bc6d248e9af96c6dbe7f319"
server_key_version: "kuduclusterkey@0"
I20260810 11:32:40.577301 20025 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786361401922362-8267-0/minicluster-data/ts-0/wal/instance:
uuid: "130ff4307aa34e638e62a7e068c3595c"
format_stamp: "Formatted at 2026-08-10 11:32:40 on dist-test-slave-srgv"
server_key: "a531a836a9e5b9b03f5b30f524636c6a"
server_key_iv: "1a8d81596bc6d248e9af96c6dbe7f319"
server_key_version: "kuduclusterkey@0"
I20260810 11:32:40.588968 20025 fs_manager.cc:691] Time spent creating directory manager: real 0.011s user 0.002s sys 0.007s
I20260810 11:32:40.631340 20046 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:32:40.634187 20025 fs_manager.cc:725] Time spent opening block manager: real 0.024s user 0.007s sys 0.015s
I20260810 11:32:40.634550 20025 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786361401922362-8267-0/minicluster-data/ts-0/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786361401922362-8267-0/minicluster-data/ts-0/wal
uuid: "130ff4307aa34e638e62a7e068c3595c"
format_stamp: "Formatted at 2026-08-10 11:32:40 on dist-test-slave-srgv"
server_key: "a531a836a9e5b9b03f5b30f524636c6a"
server_key_iv: "1a8d81596bc6d248e9af96c6dbe7f319"
server_key_version: "kuduclusterkey@0"
I20260810 11:32:40.635028 20025 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786361401922362-8267-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786361401922362-8267-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786361401922362-8267-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:32:40.664024 20025 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:32:40.672154 20025 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:32:40.673030 20025 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:32:40.676359 20025 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:32:40.680050 20025 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:32:40.680398 20025 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260810 11:32:40.680768 20025 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:32:40.680967 20025 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 11:32:40 dist-test-slave-srgv krb5kdc[19455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361560, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.254@KRBTEST.COM for HTTP/127.8.18.212@KRBTEST.COM
I20260810 11:32:40.968019 20025 rpc_server.cc:307] RPC server started. Bound to: 127.8.18.193:35461
I20260810 11:32:40.967871 20161 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.18.193:35461 every 8 connection(s)
I20260810 11:32:40.970927 20025 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786361401922362-8267-0/minicluster-data/ts-0/data/info.pb
I20260810 11:32:40.978304 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 20025
I20260810 11:32:40.978566 8267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786361401922362-8267-0/minicluster-data/ts-0/wal/instance
I20260810 11:32:40.979656 8267 external_mini_cluster.cc:1428] Setting key 8f1b821c83cf939a15711adf0e494640
Aug 10 11:32:41 dist-test-slave-srgv krb5kdc[19455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361560, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.193@KRBTEST.COM for kudu/127.8.18.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.8.18.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.18.194@KRBTEST.COM" created.
I20260810 11:32:41.043404 20164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:32:40.977934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.193:38623 (local address 127.8.18.254:45021)
0810 11:32:41.006362 (+ 28428us) server_negotiation.cc:206] Beginning negotiation
0810 11:32:41.006374 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 11:32:41.006402 (+ 28us) server_negotiation.cc:407] Connection header received
0810 11:32:41.006486 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:32:41.006494 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:32:41.006609 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:32:41.006780 (+ 171us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:32:41.010905 (+ 4125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:32:41.012992 (+ 2087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:32:41.017708 (+ 4716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:32:41.019278 (+ 1570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:32:41.019466 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:32:41.029371 (+ 9905us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:32:41.029382 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:32:41.029388 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:32:41.029488 (+ 100us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:32:41.036394 (+ 6906us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:32:41.038154 (+ 1760us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:32:41.038162 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:32:41.038165 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:32:41.038496 (+ 331us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:32:41.039716 (+ 1220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:32:41.039727 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:32:41.039734 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:32:41.040594 (+ 860us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:32:41.040794 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0810 11:32:41.042273 (+ 1479us) server_negotiation.cc:299] Negotiation successful
0810 11:32:41.042727 (+ 454us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28174,"thread_start_us":27908,"threads_started":1}
I20260810 11:32:41.046728 20165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:32:40.976812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:45021 (local address 127.8.18.193:38623)
0810 11:32:40.999122 (+ 22310us) negotiation.cc:107] Waiting for socket to connect
0810 11:32:40.999245 (+ 123us) client_negotiation.cc:174] Beginning negotiation
0810 11:32:41.004564 (+ 5319us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:32:41.007528 (+ 2964us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:32:41.007548 (+ 20us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:32:41.009049 (+ 1501us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:32:41.010556 (+ 1507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:32:41.010597 (+ 41us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:32:41.013441 (+ 2844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:32:41.013452 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:32:41.017356 (+ 3904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:32:41.017388 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:32:41.020428 (+ 3040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:32:41.020456 (+ 28us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:32:41.021098 (+ 642us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:32:41.022655 (+ 1557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:32:41.022747 (+ 92us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:32:41.028733 (+ 5986us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:32:41.036844 (+ 8111us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:32:41.036854 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:32:41.036871 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:32:41.037920 (+ 1049us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:32:41.038836 (+ 916us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:32:41.038845 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:32:41.038852 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:32:41.039437 (+ 585us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:32:41.040946 (+ 1509us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:32:41.040961 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:32:41.041678 (+ 717us) client_negotiation.cc:769] Sending connection context
0810 11:32:41.043967 (+ 2289us) client_negotiation.cc:240] Negotiation successful
0810 11:32:41.044853 (+ 886us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21734,"thread_start_us":21297,"threads_started":1}
I20260810 11:32:41.048934 20162 heartbeater.cc:347] Connected to a master server at 127.8.18.254:45021
I20260810 11:32:41.049320 20162 heartbeater.cc:464] Registering TS with master...
I20260810 11:32:41.050375 20162 heartbeater.cc:511] Master 127.8.18.254:45021 requested a full tablet report, sending...
I20260810 11:32:41.053361 19940 ts_manager.cc:194] Registered new tserver with Master: 130ff4307aa34e638e62a7e068c3595c (127.8.18.193:35461)
I20260810 11:32:41.055639 19940 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.18.193@KRBTEST.COM'} at 127.8.18.193:38623
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.18.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.194.keytab.
Entry for principal kudu/127.8.18.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.194.keytab.
I20260810 11:32:41.077910 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786361401922362-8267-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786361401922362-8267-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786361401922362-8267-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786361401922362-8267-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.18.212:36593/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.18.194:0
--local_ip_for_outbound_sockets=127.8.18.194
--webserver_interface=127.8.18.194
--webserver_port=0
--tserver_master_addrs=127.8.18.254:45021
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.194.keytab
--principal=kudu/127.8.18.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.18.212:43951
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786361401922362-8267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786361401922362-8267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786361401922362-8267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 11:32:41 dist-test-slave-srgv krb5kdc[19455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361560, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.254@KRBTEST.COM for HTTP/127.8.18.212@KRBTEST.COM
W20260810 11:32:41.843840 20168 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:32:41.844476 20168 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:32:41.844830 20168 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:32:41.861541 20168 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:32:41.861876 20168 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:32:41.862197 20168 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.18.194
I20260810 11:32:41.882259 20168 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.18.212:43951
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786361401922362-8267-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786361401922362-8267-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.8.18.212:36593/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.194.keytab
--principal=kudu/127.8.18.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.18.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786361401922362-8267-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.18.194
--webserver_port=0
--tserver_master_addrs=127.8.18.254:45021
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.18.194
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786361401922362-8267-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:32:41.885706 20168 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:32:41.888633 20168 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:32:41.915441 20182 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:32:41.915382 20183 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:32:41.916921 20185 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:32:41.918705 20168 server_base.cc:1034] running on GCE node
I20260810 11:32:41.919872 20168 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:32:41.921386 20168 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:32:41.922802 20168 hybrid_clock.cc:648] HybridClock initialized: now 1786361561922700 us; error 74 us; skew 500 ppm
I20260810 11:32:41.923429 20168 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:32:41 dist-test-slave-srgv krb5kdc[19455](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361561, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:32:41.930641 20168 init.cc:377] Logged in from keytab as kudu/127.8.18.194@KRBTEST.COM (short username kudu)
I20260810 11:32:41.935065 20168 webserver.cc:460] Webserver started at http://127.8.18.194:39675/ using document root <none> and password file <none>
I20260810 11:32:41.937148 20168 fs_manager.cc:359] Metadata directory not provided
I20260810 11:32:41.937639 20168 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:32:41.938483 20168 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 11:32:41 dist-test-slave-srgv krb5kdc[19455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361561, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.194@KRBTEST.COM for HTTP/127.8.18.212@KRBTEST.COM
I20260810 11:32:41.961046 20168 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786361401922362-8267-0/minicluster-data/ts-1/data/instance:
uuid: "ad79851e8a374423b7967dbf87bdcb9b"
format_stamp: "Formatted at 2026-08-10 11:32:41 on dist-test-slave-srgv"
server_key: "4d7245a81ba2b2a97acf95485c8bf00b"
server_key_iv: "9c931c4e10c658d3fea5ae37fd2cf421"
server_key_version: "kuduclusterkey@0"
I20260810 11:32:41.962441 20168 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786361401922362-8267-0/minicluster-data/ts-1/wal/instance:
uuid: "ad79851e8a374423b7967dbf87bdcb9b"
format_stamp: "Formatted at 2026-08-10 11:32:41 on dist-test-slave-srgv"
server_key: "4d7245a81ba2b2a97acf95485c8bf00b"
server_key_iv: "9c931c4e10c658d3fea5ae37fd2cf421"
server_key_version: "kuduclusterkey@0"
I20260810 11:32:41.973743 20168 fs_manager.cc:691] Time spent creating directory manager: real 0.010s user 0.009s sys 0.000s
I20260810 11:32:42.000139 20192 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:32:42.002791 20168 fs_manager.cc:725] Time spent opening block manager: real 0.009s user 0.006s sys 0.000s
I20260810 11:32:42.003122 20168 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786361401922362-8267-0/minicluster-data/ts-1/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786361401922362-8267-0/minicluster-data/ts-1/wal
uuid: "ad79851e8a374423b7967dbf87bdcb9b"
format_stamp: "Formatted at 2026-08-10 11:32:41 on dist-test-slave-srgv"
server_key: "4d7245a81ba2b2a97acf95485c8bf00b"
server_key_iv: "9c931c4e10c658d3fea5ae37fd2cf421"
server_key_version: "kuduclusterkey@0"
I20260810 11:32:42.003500 20168 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786361401922362-8267-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786361401922362-8267-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786361401922362-8267-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:32:42.043576 20168 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:32:42.052913 20168 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:32:42.053804 20168 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:32:42.056124 20168 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:32:42.059721 20168 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:32:42.059959 20168 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260810 11:32:42.060279 20168 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:32:42.060470 20168 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:32:42.060603 20162 heartbeater.cc:503] Master 127.8.18.254:45021 was elected leader, sending a full tablet report...
I20260810 11:32:42.226670 20168 rpc_server.cc:307] RPC server started. Bound to: 127.8.18.194:46579
I20260810 11:32:42.226759 20307 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.18.194:46579 every 8 connection(s)
I20260810 11:32:42.228750 20168 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786361401922362-8267-0/minicluster-data/ts-1/data/info.pb
I20260810 11:32:42.234057 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 20168
I20260810 11:32:42.234269 8267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786361401922362-8267-0/minicluster-data/ts-1/wal/instance
I20260810 11:32:42.234918 8267 external_mini_cluster.cc:1428] Setting key 67586f823188988350e5bf6276a1da21
WARNING: no policy specified for kudu/127.8.18.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.18.195@KRBTEST.COM" created.
Aug 10 11:32:42 dist-test-slave-srgv krb5kdc[19455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361561, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.194@KRBTEST.COM for kudu/127.8.18.254@KRBTEST.COM
I20260810 11:32:42.305796 20311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:32:42.236396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:45021 (local address 127.8.18.194:40313)
0810 11:32:42.243259 (+ 6863us) negotiation.cc:107] Waiting for socket to connect
0810 11:32:42.243334 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0810 11:32:42.257970 (+ 14636us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:32:42.269800 (+ 11830us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:32:42.269818 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:32:42.271053 (+ 1235us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:32:42.272697 (+ 1644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:32:42.272740 (+ 43us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:32:42.275236 (+ 2496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:32:42.275242 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:32:42.278895 (+ 3653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:32:42.278968 (+ 73us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:32:42.281532 (+ 2564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:32:42.281545 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:32:42.282029 (+ 484us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:32:42.288233 (+ 6204us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:32:42.288274 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:32:42.291555 (+ 3281us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:32:42.298145 (+ 6590us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:32:42.298158 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:32:42.298180 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:32:42.299272 (+ 1092us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:32:42.300147 (+ 875us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:32:42.300152 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:32:42.300155 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:32:42.300621 (+ 466us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:32:42.301627 (+ 1006us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:32:42.301640 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:32:42.302362 (+ 722us) client_negotiation.cc:769] Sending connection context
0810 11:32:42.302781 (+ 419us) client_negotiation.cc:240] Negotiation successful
0810 11:32:42.303531 (+ 750us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6356,"thread_start_us":16890,"threads_started":1}
I20260810 11:32:42.308799 20310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:32:42.235764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.194:40313 (local address 127.8.18.254:45021)
0810 11:32:42.242252 (+ 6488us) server_negotiation.cc:206] Beginning negotiation
0810 11:32:42.242260 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:32:42.253529 (+ 11269us) server_negotiation.cc:407] Connection header received
0810 11:32:42.268404 (+ 14875us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:32:42.268412 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:32:42.268541 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:32:42.268670 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:32:42.273061 (+ 4391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:32:42.274914 (+ 1853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:32:42.279383 (+ 4469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:32:42.280559 (+ 1176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:32:42.280717 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:32:42.291950 (+ 11233us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:32:42.291959 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:32:42.291964 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:32:42.292030 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:32:42.297842 (+ 5812us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:32:42.299524 (+ 1682us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:32:42.299531 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:32:42.299535 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:32:42.299964 (+ 429us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:32:42.300821 (+ 857us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:32:42.300826 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:32:42.300829 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:32:42.301380 (+ 551us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:32:42.307406 (+ 6026us) server_negotiation.cc:1035] Waiting for connection context
0810 11:32:42.307603 (+ 197us) server_negotiation.cc:299] Negotiation successful
0810 11:32:42.308009 (+ 406us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6277,"thread_start_us":6072,"threads_started":1}
I20260810 11:32:42.311044 20308 heartbeater.cc:347] Connected to a master server at 127.8.18.254:45021
I20260810 11:32:42.311494 20308 heartbeater.cc:464] Registering TS with master...
I20260810 11:32:42.312808 20308 heartbeater.cc:511] Master 127.8.18.254:45021 requested a full tablet report, sending...
I20260810 11:32:42.315737 19940 ts_manager.cc:194] Registered new tserver with Master: ad79851e8a374423b7967dbf87bdcb9b (127.8.18.194:46579)
I20260810 11:32:42.317198 19940 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.18.194@KRBTEST.COM'} at 127.8.18.194:40313
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.18.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.195.keytab.
Entry for principal kudu/127.8.18.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.195.keytab.
I20260810 11:32:42.336493 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786361401922362-8267-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786361401922362-8267-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786361401922362-8267-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786361401922362-8267-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.18.212:36593/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.18.195:0
--local_ip_for_outbound_sockets=127.8.18.195
--webserver_interface=127.8.18.195
--webserver_port=0
--tserver_master_addrs=127.8.18.254:45021
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.195.keytab
--principal=kudu/127.8.18.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.18.212:43951
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786361401922362-8267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786361401922362-8267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786361401922362-8267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:32:43.009155 20314 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:32:43.009617 20314 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:32:43.009835 20314 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:32:43.019459 20314 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:32:43.019733 20314 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:32:43.019948 20314 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.18.195
I20260810 11:32:43.032778 20314 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.18.212:43951
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786361401922362-8267-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786361401922362-8267-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.8.18.212:36593/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.195.keytab
--principal=kudu/127.8.18.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.18.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786361401922362-8267-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.18.195
--webserver_port=0
--tserver_master_addrs=127.8.18.254:45021
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.18.195
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786361401922362-8267-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:32:43.034394 20314 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:32:43.035944 20314 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:32:43.054059 20319 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:32:43.054158 20323 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:32:43.054495 20321 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:32:43.055712 20314 server_base.cc:1034] running on GCE node
I20260810 11:32:43.056478 20314 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:32:43.058019 20314 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:32:43.059407 20314 hybrid_clock.cc:648] HybridClock initialized: now 1786361563059325 us; error 52 us; skew 500 ppm
I20260810 11:32:43.059886 20314 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:32:43 dist-test-slave-srgv krb5kdc[19455](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361563, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:32:43.064785 20314 init.cc:377] Logged in from keytab as kudu/127.8.18.195@KRBTEST.COM (short username kudu)
I20260810 11:32:43.067134 20314 webserver.cc:460] Webserver started at http://127.8.18.195:35873/ using document root <none> and password file <none>
I20260810 11:32:43.067935 20314 fs_manager.cc:359] Metadata directory not provided
I20260810 11:32:43.068065 20314 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:32:43.068370 20314 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 11:32:43 dist-test-slave-srgv krb5kdc[19455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361563, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.195@KRBTEST.COM for HTTP/127.8.18.212@KRBTEST.COM
I20260810 11:32:43.091063 20314 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786361401922362-8267-0/minicluster-data/ts-2/data/instance:
uuid: "380fcbc24f8f4b78b16632f6a1c3687e"
format_stamp: "Formatted at 2026-08-10 11:32:43 on dist-test-slave-srgv"
server_key: "4c2e6bc08f7daeb35d3b3c364c26829f"
server_key_iv: "de9b9532640254b924366b96bfed3ccb"
server_key_version: "kuduclusterkey@0"
I20260810 11:32:43.092595 20314 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786361401922362-8267-0/minicluster-data/ts-2/wal/instance:
uuid: "380fcbc24f8f4b78b16632f6a1c3687e"
format_stamp: "Formatted at 2026-08-10 11:32:43 on dist-test-slave-srgv"
server_key: "4c2e6bc08f7daeb35d3b3c364c26829f"
server_key_iv: "de9b9532640254b924366b96bfed3ccb"
server_key_version: "kuduclusterkey@0"
I20260810 11:32:43.100778 20314 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.001s sys 0.009s
I20260810 11:32:43.120960 20331 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:32:43.123143 20314 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260810 11:32:43.123371 20314 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786361401922362-8267-0/minicluster-data/ts-2/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786361401922362-8267-0/minicluster-data/ts-2/wal
uuid: "380fcbc24f8f4b78b16632f6a1c3687e"
format_stamp: "Formatted at 2026-08-10 11:32:43 on dist-test-slave-srgv"
server_key: "4c2e6bc08f7daeb35d3b3c364c26829f"
server_key_iv: "de9b9532640254b924366b96bfed3ccb"
server_key_version: "kuduclusterkey@0"
I20260810 11:32:43.123624 20314 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786361401922362-8267-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786361401922362-8267-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786361401922362-8267-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:32:43.147868 20314 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:32:43.153352 20314 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:32:43.153808 20314 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:32:43.155093 20314 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:32:43.157102 20314 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:32:43.157189 20314 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:32:43.157327 20314 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:32:43.157505 20314 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:32:43.226912 20314 rpc_server.cc:307] RPC server started. Bound to: 127.8.18.195:46491
I20260810 11:32:43.226943 20447 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.18.195:46491 every 8 connection(s)
I20260810 11:32:43.228971 20314 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786361401922362-8267-0/minicluster-data/ts-2/data/info.pb
I20260810 11:32:43.237577 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 20314
I20260810 11:32:43.237841 8267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786361401922362-8267-0/minicluster-data/ts-2/wal/instance
I20260810 11:32:43.238541 8267 external_mini_cluster.cc:1428] Setting key 660441eaa55784997711161c660ca8b5
Aug 10 11:32:43 dist-test-slave-srgv krb5kdc[19455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361563, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.195@KRBTEST.COM for kudu/127.8.18.254@KRBTEST.COM
I20260810 11:32:43.258931 20450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:32:43.232621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.195:44005 (local address 127.8.18.254:45021)
0810 11:32:43.233532 (+ 911us) server_negotiation.cc:206] Beginning negotiation
0810 11:32:43.233539 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:32:43.235002 (+ 1463us) server_negotiation.cc:407] Connection header received
0810 11:32:43.237913 (+ 2911us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:32:43.237923 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:32:43.238089 (+ 166us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:32:43.238251 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:32:43.240891 (+ 2640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:32:43.242189 (+ 1298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:32:43.244567 (+ 2378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:32:43.245302 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:32:43.245508 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:32:43.250800 (+ 5292us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:32:43.250806 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:32:43.250810 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:32:43.250853 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:32:43.254928 (+ 4075us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:32:43.256194 (+ 1266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:32:43.256198 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:32:43.256201 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:32:43.256397 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:32:43.256953 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:32:43.256956 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:32:43.256958 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:32:43.257319 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:32:43.257529 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0810 11:32:43.258382 (+ 853us) server_negotiation.cc:299] Negotiation successful
0810 11:32:43.258597 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":735,"thread_start_us":478,"threads_started":1}
I20260810 11:32:43.260488 20451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:32:43.233560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:45021 (local address 127.8.18.195:44005)
0810 11:32:43.234794 (+ 1234us) negotiation.cc:107] Waiting for socket to connect
0810 11:32:43.234844 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0810 11:32:43.237512 (+ 2668us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:32:43.238579 (+ 1067us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:32:43.238625 (+ 46us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:32:43.239375 (+ 750us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:32:43.240671 (+ 1296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:32:43.240707 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:32:43.242499 (+ 1792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:32:43.242506 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:32:43.244279 (+ 1773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:32:43.244298 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:32:43.245803 (+ 1505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:32:43.245814 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:32:43.246212 (+ 398us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:32:43.247312 (+ 1100us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:32:43.247342 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:32:43.250463 (+ 3121us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:32:43.255167 (+ 4704us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:32:43.255180 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:32:43.255206 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:32:43.256066 (+ 860us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:32:43.256511 (+ 445us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:32:43.256515 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:32:43.256518 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:32:43.256843 (+ 325us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:32:43.257548 (+ 705us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:32:43.257557 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:32:43.257940 (+ 383us) client_negotiation.cc:769] Sending connection context
0810 11:32:43.258250 (+ 310us) client_negotiation.cc:240] Negotiation successful
0810 11:32:43.258783 (+ 533us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":909,"thread_start_us":599,"threads_started":1}
I20260810 11:32:43.262696 20448 heartbeater.cc:347] Connected to a master server at 127.8.18.254:45021
I20260810 11:32:43.263222 20448 heartbeater.cc:464] Registering TS with master...
I20260810 11:32:43.264237 20448 heartbeater.cc:511] Master 127.8.18.254:45021 requested a full tablet report, sending...
I20260810 11:32:43.266366 19940 ts_manager.cc:194] Registered new tserver with Master: 380fcbc24f8f4b78b16632f6a1c3687e (127.8.18.195:46491)
I20260810 11:32:43.267731 19940 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.18.195@KRBTEST.COM'} at 127.8.18.195:44005
I20260810 11:32:43.268975 8267 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 11:32:43.292817 20450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:32:43.275128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43476 (local address 127.8.18.254:45021)
0810 11:32:43.275335 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0810 11:32:43.275340 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:32:43.276009 (+ 669us) server_negotiation.cc:407] Connection header received
0810 11:32:43.276234 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:32:43.276239 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:32:43.276324 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:32:43.276442 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:32:43.278443 (+ 2001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:32:43.279310 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:32:43.281215 (+ 1905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:32:43.281816 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:32:43.281974 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:32:43.285264 (+ 3290us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:32:43.285270 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:32:43.285274 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:32:43.285318 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:32:43.289083 (+ 3765us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:32:43.290329 (+ 1246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:32:43.290335 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:32:43.290338 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:32:43.290525 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:32:43.291030 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:32:43.291036 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:32:43.291044 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:32:43.291377 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:32:43.291737 (+ 360us) server_negotiation.cc:1035] Waiting for connection context
0810 11:32:43.292094 (+ 357us) server_negotiation.cc:299] Negotiation successful
0810 11:32:43.292412 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260810 11:32:43.299124 19940 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43476:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 11:32:43.314565 19940 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 11:32:43.322211 20308 heartbeater.cc:503] Master 127.8.18.254:45021 was elected leader, sending a full tablet report...
I20260810 11:32:43.360553 20466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:32:43.337747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49702 (local address 127.8.18.193:35461)
0810 11:32:43.342793 (+ 5046us) server_negotiation.cc:206] Beginning negotiation
0810 11:32:43.342809 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0810 11:32:43.344996 (+ 2187us) server_negotiation.cc:407] Connection header received
0810 11:32:43.345619 (+ 623us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:32:43.345634 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:32:43.346048 (+ 414us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:32:43.346403 (+ 355us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:32:43.348271 (+ 1868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:32:43.350374 (+ 2103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:32:43.355976 (+ 5602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:32:43.358013 (+ 2037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:32:43.358226 (+ 213us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:32:43.358619 (+ 393us) server_negotiation.cc:1035] Waiting for connection context
0810 11:32:43.359749 (+ 1130us) server_negotiation.cc:299] Negotiation successful
0810 11:32:43.359999 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4651,"thread_start_us":4381,"threads_started":1}
I20260810 11:32:43.360713 20465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:32:43.337170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.193:35461 (local address 127.0.0.1:49702)
0810 11:32:43.344805 (+ 7635us) negotiation.cc:107] Waiting for socket to connect
0810 11:32:43.344830 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0810 11:32:43.345224 (+ 394us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:32:43.347066 (+ 1842us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:32:43.347079 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:32:43.347190 (+ 111us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:32:43.347968 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:32:43.347988 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:32:43.350775 (+ 2787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:32:43.350781 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:32:43.355760 (+ 4979us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:32:43.355775 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:32:43.358916 (+ 3141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:32:43.358929 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:32:43.359277 (+ 348us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:32:43.359408 (+ 131us) client_negotiation.cc:769] Sending connection context
0810 11:32:43.359658 (+ 250us) client_negotiation.cc:240] Negotiation successful
0810 11:32:43.359972 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7384,"mutex_wait_us":80,"thread_start_us":7076,"threads_started":1}
I20260810 11:32:43.362967 20463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:32:43.334051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.195:46491 (local address 127.0.0.1:56744)
0810 11:32:43.336430 (+ 2379us) negotiation.cc:107] Waiting for socket to connect
0810 11:32:43.336490 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0810 11:32:43.336807 (+ 317us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:32:43.350618 (+ 13811us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:32:43.350623 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:32:43.350656 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:32:43.352252 (+ 1596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:32:43.352266 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:32:43.355252 (+ 2986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:32:43.355259 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:32:43.357997 (+ 2738us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:32:43.358014 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:32:43.361846 (+ 3832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:32:43.361853 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:32:43.362088 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:32:43.362119 (+ 31us) client_negotiation.cc:769] Sending connection context
0810 11:32:43.362252 (+ 133us) client_negotiation.cc:240] Negotiation successful
0810 11:32:43.362427 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2057,"thread_start_us":7365,"threads_started":1}
I20260810 11:32:43.363618 20462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:32:43.334578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56744 (local address 127.8.18.195:46491)
0810 11:32:43.349239 (+ 14661us) server_negotiation.cc:206] Beginning negotiation
0810 11:32:43.349254 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 11:32:43.349321 (+ 67us) server_negotiation.cc:407] Connection header received
0810 11:32:43.349517 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:32:43.349532 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:32:43.350021 (+ 489us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:32:43.350334 (+ 313us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:32:43.352588 (+ 2254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:32:43.355062 (+ 2474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:32:43.358231 (+ 3169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:32:43.361026 (+ 2795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:32:43.361249 (+ 223us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:32:43.361665 (+ 416us) server_negotiation.cc:1035] Waiting for connection context
0810 11:32:43.362313 (+ 648us) server_negotiation.cc:299] Negotiation successful
0810 11:32:43.362574 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14361,"thread_start_us":15226,"threads_started":1}
I20260810 11:32:43.365793 20461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:32:43.335164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.194:46579 (local address 127.0.0.1:50804)
0810 11:32:43.343313 (+ 8149us) negotiation.cc:107] Waiting for socket to connect
0810 11:32:43.343346 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0810 11:32:43.343614 (+ 268us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:32:43.351603 (+ 7989us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:32:43.351621 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:32:43.351660 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:32:43.354407 (+ 2747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:32:43.354424 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:32:43.357149 (+ 2725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:32:43.357155 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:32:43.361801 (+ 4646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:32:43.361830 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:32:43.364412 (+ 2582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:32:43.364421 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:32:43.364654 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:32:43.364707 (+ 53us) client_negotiation.cc:769] Sending connection context
0810 11:32:43.364822 (+ 115us) client_negotiation.cc:240] Negotiation successful
0810 11:32:43.364992 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7916,"mutex_wait_us":107,"thread_start_us":658,"threads_started":1}
I20260810 11:32:43.366328 20095 tablet_service.cc:1467] Processing CreateTablet for tablet 551766b41c5340c599baf7232992f771 (DEFAULT_TABLE table=test-table [id=fed626aa9a36405e9e9e68421ba26faa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:32:43.367172 20464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:32:43.335300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50804 (local address 127.8.18.194:46579)
0810 11:32:43.350810 (+ 15510us) server_negotiation.cc:206] Beginning negotiation
0810 11:32:43.350821 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 11:32:43.350853 (+ 32us) server_negotiation.cc:407] Connection header received
0810 11:32:43.350954 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:32:43.350963 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:32:43.351306 (+ 343us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:32:43.351594 (+ 288us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:32:43.354669 (+ 3075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:32:43.356840 (+ 2171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:32:43.362038 (+ 5198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:32:43.364079 (+ 2041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:32:43.364268 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:32:43.364559 (+ 291us) server_negotiation.cc:1035] Waiting for connection context
0810 11:32:43.364917 (+ 358us) server_negotiation.cc:299] Negotiation successful
0810 11:32:43.366692 (+ 1775us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15305,"thread_start_us":15027,"threads_started":1}
I20260810 11:32:43.368373 20095 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 551766b41c5340c599baf7232992f771. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:32:43.368388 20381 tablet_service.cc:1467] Processing CreateTablet for tablet 551766b41c5340c599baf7232992f771 (DEFAULT_TABLE table=test-table [id=fed626aa9a36405e9e9e68421ba26faa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:32:43.371073 20381 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 551766b41c5340c599baf7232992f771. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:32:43.372471 20242 tablet_service.cc:1467] Processing CreateTablet for tablet 551766b41c5340c599baf7232992f771 (DEFAULT_TABLE table=test-table [id=fed626aa9a36405e9e9e68421ba26faa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:32:43.375013 20242 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 551766b41c5340c599baf7232992f771. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:32:43.385914 20467 tablet_bootstrap.cc:492] T 551766b41c5340c599baf7232992f771 P 130ff4307aa34e638e62a7e068c3595c: Bootstrap starting.
I20260810 11:32:43.390659 20467 tablet_bootstrap.cc:654] T 551766b41c5340c599baf7232992f771 P 130ff4307aa34e638e62a7e068c3595c: Neither blocks nor log segments found. Creating new log.
I20260810 11:32:43.392400 20467 log.cc:824] T 551766b41c5340c599baf7232992f771 P 130ff4307aa34e638e62a7e068c3595c: Log is configured to *not* fsync() on all Append() calls
I20260810 11:32:43.395324 20469 tablet_bootstrap.cc:492] T 551766b41c5340c599baf7232992f771 P 380fcbc24f8f4b78b16632f6a1c3687e: Bootstrap starting.
I20260810 11:32:43.397480 20470 tablet_bootstrap.cc:492] T 551766b41c5340c599baf7232992f771 P ad79851e8a374423b7967dbf87bdcb9b: Bootstrap starting.
I20260810 11:32:43.399143 20467 tablet_bootstrap.cc:492] T 551766b41c5340c599baf7232992f771 P 130ff4307aa34e638e62a7e068c3595c: No bootstrap required, opened a new log
I20260810 11:32:43.399573 20467 ts_tablet_manager.cc:1397] T 551766b41c5340c599baf7232992f771 P 130ff4307aa34e638e62a7e068c3595c: Time spent bootstrapping tablet: real 0.014s user 0.007s sys 0.003s
I20260810 11:32:43.401032 20469 tablet_bootstrap.cc:654] T 551766b41c5340c599baf7232992f771 P 380fcbc24f8f4b78b16632f6a1c3687e: Neither blocks nor log segments found. Creating new log.
I20260810 11:32:43.403118 20470 tablet_bootstrap.cc:654] T 551766b41c5340c599baf7232992f771 P ad79851e8a374423b7967dbf87bdcb9b: Neither blocks nor log segments found. Creating new log.
I20260810 11:32:43.403400 20469 log.cc:824] T 551766b41c5340c599baf7232992f771 P 380fcbc24f8f4b78b16632f6a1c3687e: Log is configured to *not* fsync() on all Append() calls
I20260810 11:32:43.405203 20470 log.cc:824] T 551766b41c5340c599baf7232992f771 P ad79851e8a374423b7967dbf87bdcb9b: Log is configured to *not* fsync() on all Append() calls
I20260810 11:32:43.408244 20469 tablet_bootstrap.cc:492] T 551766b41c5340c599baf7232992f771 P 380fcbc24f8f4b78b16632f6a1c3687e: No bootstrap required, opened a new log
I20260810 11:32:43.408717 20469 ts_tablet_manager.cc:1397] T 551766b41c5340c599baf7232992f771 P 380fcbc24f8f4b78b16632f6a1c3687e: Time spent bootstrapping tablet: real 0.014s user 0.005s sys 0.005s
I20260810 11:32:43.414333 20470 tablet_bootstrap.cc:492] T 551766b41c5340c599baf7232992f771 P ad79851e8a374423b7967dbf87bdcb9b: No bootstrap required, opened a new log
I20260810 11:32:43.414971 20470 ts_tablet_manager.cc:1397] T 551766b41c5340c599baf7232992f771 P ad79851e8a374423b7967dbf87bdcb9b: Time spent bootstrapping tablet: real 0.018s user 0.009s sys 0.003s
I20260810 11:32:43.414935 20467 raft_consensus.cc:348] T 551766b41c5340c599baf7232992f771 P 130ff4307aa34e638e62a7e068c3595c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "380fcbc24f8f4b78b16632f6a1c3687e" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 46491 } } peers { permanent_uuid: "ad79851e8a374423b7967dbf87bdcb9b" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 46579 } } peers { permanent_uuid: "130ff4307aa34e638e62a7e068c3595c" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 35461 } }
I20260810 11:32:43.415778 20467 raft_consensus.cc:374] T 551766b41c5340c599baf7232992f771 P 130ff4307aa34e638e62a7e068c3595c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:32:43.415977 20467 raft_consensus.cc:729] T 551766b41c5340c599baf7232992f771 P 130ff4307aa34e638e62a7e068c3595c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 130ff4307aa34e638e62a7e068c3595c, State: Initialized, Role: FOLLOWER
I20260810 11:32:43.417205 20467 consensus_queue.cc:260] T 551766b41c5340c599baf7232992f771 P 130ff4307aa34e638e62a7e068c3595c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "380fcbc24f8f4b78b16632f6a1c3687e" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 46491 } } peers { permanent_uuid: "ad79851e8a374423b7967dbf87bdcb9b" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 46579 } } peers { permanent_uuid: "130ff4307aa34e638e62a7e068c3595c" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 35461 } }
I20260810 11:32:43.423818 20467 ts_tablet_manager.cc:1428] T 551766b41c5340c599baf7232992f771 P 130ff4307aa34e638e62a7e068c3595c: Time spent starting tablet: real 0.024s user 0.006s sys 0.017s
I20260810 11:32:43.427416 20469 raft_consensus.cc:348] T 551766b41c5340c599baf7232992f771 P 380fcbc24f8f4b78b16632f6a1c3687e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "380fcbc24f8f4b78b16632f6a1c3687e" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 46491 } } peers { permanent_uuid: "ad79851e8a374423b7967dbf87bdcb9b" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 46579 } } peers { permanent_uuid: "130ff4307aa34e638e62a7e068c3595c" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 35461 } }
I20260810 11:32:43.428143 20469 raft_consensus.cc:374] T 551766b41c5340c599baf7232992f771 P 380fcbc24f8f4b78b16632f6a1c3687e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:32:43.428436 20469 raft_consensus.cc:729] T 551766b41c5340c599baf7232992f771 P 380fcbc24f8f4b78b16632f6a1c3687e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 380fcbc24f8f4b78b16632f6a1c3687e, State: Initialized, Role: FOLLOWER
I20260810 11:32:43.429565 20469 consensus_queue.cc:260] T 551766b41c5340c599baf7232992f771 P 380fcbc24f8f4b78b16632f6a1c3687e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "380fcbc24f8f4b78b16632f6a1c3687e" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 46491 } } peers { permanent_uuid: "ad79851e8a374423b7967dbf87bdcb9b" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 46579 } } peers { permanent_uuid: "130ff4307aa34e638e62a7e068c3595c" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 35461 } }
I20260810 11:32:43.431128 20470 raft_consensus.cc:348] T 551766b41c5340c599baf7232992f771 P ad79851e8a374423b7967dbf87bdcb9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "380fcbc24f8f4b78b16632f6a1c3687e" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 46491 } } peers { permanent_uuid: "ad79851e8a374423b7967dbf87bdcb9b" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 46579 } } peers { permanent_uuid: "130ff4307aa34e638e62a7e068c3595c" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 35461 } }
I20260810 11:32:43.431752 20470 raft_consensus.cc:374] T 551766b41c5340c599baf7232992f771 P ad79851e8a374423b7967dbf87bdcb9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:32:43.431910 20470 raft_consensus.cc:729] T 551766b41c5340c599baf7232992f771 P ad79851e8a374423b7967dbf87bdcb9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad79851e8a374423b7967dbf87bdcb9b, State: Initialized, Role: FOLLOWER
I20260810 11:32:43.432672 20470 consensus_queue.cc:260] T 551766b41c5340c599baf7232992f771 P ad79851e8a374423b7967dbf87bdcb9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "380fcbc24f8f4b78b16632f6a1c3687e" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 46491 } } peers { permanent_uuid: "ad79851e8a374423b7967dbf87bdcb9b" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 46579 } } peers { permanent_uuid: "130ff4307aa34e638e62a7e068c3595c" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 35461 } }
I20260810 11:32:43.434774 20448 heartbeater.cc:503] Master 127.8.18.254:45021 was elected leader, sending a full tablet report...
I20260810 11:32:43.435137 20470 ts_tablet_manager.cc:1428] T 551766b41c5340c599baf7232992f771 P ad79851e8a374423b7967dbf87bdcb9b: Time spent starting tablet: real 0.020s user 0.011s sys 0.007s
I20260810 11:32:43.437526 20469 ts_tablet_manager.cc:1428] T 551766b41c5340c599baf7232992f771 P 380fcbc24f8f4b78b16632f6a1c3687e: Time spent starting tablet: real 0.029s user 0.010s sys 0.015s
W20260810 11:32:43.482012 20449 tablet.cc:2369] T 551766b41c5340c599baf7232992f771 P 380fcbc24f8f4b78b16632f6a1c3687e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 11:32:43.488776 20309 tablet.cc:2369] T 551766b41c5340c599baf7232992f771 P ad79851e8a374423b7967dbf87bdcb9b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 11:32:43.498313 20163 tablet.cc:2369] T 551766b41c5340c599baf7232992f771 P 130ff4307aa34e638e62a7e068c3595c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 11:32:43.567490 20475 raft_consensus.cc:482] T 551766b41c5340c599baf7232992f771 P ad79851e8a374423b7967dbf87bdcb9b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 11:32:43.568013 20475 raft_consensus.cc:504] T 551766b41c5340c599baf7232992f771 P ad79851e8a374423b7967dbf87bdcb9b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "380fcbc24f8f4b78b16632f6a1c3687e" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 46491 } } peers { permanent_uuid: "ad79851e8a374423b7967dbf87bdcb9b" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 46579 } } peers { permanent_uuid: "130ff4307aa34e638e62a7e068c3595c" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 35461 } }
I20260810 11:32:43.570290 20475 leader_election.cc:288] T 551766b41c5340c599baf7232992f771 P ad79851e8a374423b7967dbf87bdcb9b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 380fcbc24f8f4b78b16632f6a1c3687e (127.8.18.195:46491), 130ff4307aa34e638e62a7e068c3595c (127.8.18.193:35461)
I20260810 11:32:43.581307 20462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:32:43.571200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.194:44889 (local address 127.8.18.195:46491)
0810 11:32:43.571499 (+ 299us) server_negotiation.cc:206] Beginning negotiation
0810 11:32:43.571508 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 11:32:43.572512 (+ 1004us) server_negotiation.cc:407] Connection header received
0810 11:32:43.572747 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:32:43.572753 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:32:43.572846 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:32:43.572958 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:32:43.574080 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:32:43.575364 (+ 1284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:32:43.578498 (+ 3134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:32:43.579835 (+ 1337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:32:43.579995 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:32:43.580058 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0810 11:32:43.580782 (+ 724us) server_negotiation.cc:299] Negotiation successful
0810 11:32:43.580973 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260810 11:32:43.582052 20476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:32:43.570451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.195:46491 (local address 127.8.18.194:44889)
0810 11:32:43.572341 (+ 1890us) negotiation.cc:107] Waiting for socket to connect
0810 11:32:43.572372 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0810 11:32:43.572581 (+ 209us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:32:43.572990 (+ 409us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:32:43.572995 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:32:43.573025 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:32:43.573904 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:32:43.573917 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:32:43.575514 (+ 1597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:32:43.575520 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:32:43.578236 (+ 2716us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:32:43.578256 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:32:43.580239 (+ 1983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:32:43.580249 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:32:43.580494 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:32:43.580567 (+ 73us) client_negotiation.cc:769] Sending connection context
0810 11:32:43.580698 (+ 131us) client_negotiation.cc:240] Negotiation successful
0810 11:32:43.580881 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1661,"mutex_wait_us":93,"thread_start_us":1165,"threads_started":1}
I20260810 11:32:43.582438 20466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:32:43.571154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.194:60385 (local address 127.8.18.193:35461)
0810 11:32:43.571504 (+ 350us) server_negotiation.cc:206] Beginning negotiation
0810 11:32:43.571513 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 11:32:43.572747 (+ 1234us) server_negotiation.cc:407] Connection header received
0810 11:32:43.572927 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:32:43.572935 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:32:43.573051 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:32:43.573169 (+ 118us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:32:43.574854 (+ 1685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:32:43.575999 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:32:43.579262 (+ 3263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:32:43.580686 (+ 1424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:32:43.580864 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:32:43.580935 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0810 11:32:43.581946 (+ 1011us) server_negotiation.cc:299] Negotiation successful
0810 11:32:43.582095 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103}
I20260810 11:32:43.582662 20477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:32:43.570634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.193:35461 (local address 127.8.18.194:60385)
0810 11:32:43.572579 (+ 1945us) negotiation.cc:107] Waiting for socket to connect
0810 11:32:43.572594 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 11:32:43.572768 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:32:43.573241 (+ 473us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:32:43.573249 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:32:43.573280 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:32:43.574577 (+ 1297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:32:43.574593 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:32:43.576202 (+ 1609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:32:43.576205 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:32:43.579023 (+ 2818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:32:43.579042 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:32:43.581094 (+ 2052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:32:43.581102 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:32:43.581528 (+ 426us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:32:43.581568 (+ 40us) client_negotiation.cc:769] Sending connection context
0810 11:32:43.581912 (+ 344us) client_negotiation.cc:240] Negotiation successful
0810 11:32:43.582069 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1768,"thread_start_us":1544,"threads_started":1}
I20260810 11:32:43.583670 20401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "551766b41c5340c599baf7232992f771" candidate_uuid: "ad79851e8a374423b7967dbf87bdcb9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "380fcbc24f8f4b78b16632f6a1c3687e" is_pre_election: true
I20260810 11:32:43.584033 20115 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "551766b41c5340c599baf7232992f771" candidate_uuid: "ad79851e8a374423b7967dbf87bdcb9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130ff4307aa34e638e62a7e068c3595c" is_pre_election: true
I20260810 11:32:43.584530 20401 raft_consensus.cc:2454] T 551766b41c5340c599baf7232992f771 P 380fcbc24f8f4b78b16632f6a1c3687e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad79851e8a374423b7967dbf87bdcb9b in term 0.
I20260810 11:32:43.584842 20115 raft_consensus.cc:2454] T 551766b41c5340c599baf7232992f771 P 130ff4307aa34e638e62a7e068c3595c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad79851e8a374423b7967dbf87bdcb9b in term 0.
I20260810 11:32:43.585947 20195 leader_election.cc:302] T 551766b41c5340c599baf7232992f771 P ad79851e8a374423b7967dbf87bdcb9b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 380fcbc24f8f4b78b16632f6a1c3687e, ad79851e8a374423b7967dbf87bdcb9b; no voters:
I20260810 11:32:43.586588 20475 raft_consensus.cc:2781] T 551766b41c5340c599baf7232992f771 P ad79851e8a374423b7967dbf87bdcb9b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 11:32:43.586884 20475 raft_consensus.cc:482] T 551766b41c5340c599baf7232992f771 P ad79851e8a374423b7967dbf87bdcb9b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 11:32:43.587010 20475 raft_consensus.cc:3037] T 551766b41c5340c599baf7232992f771 P ad79851e8a374423b7967dbf87bdcb9b [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:32:43.590515 20475 raft_consensus.cc:504] T 551766b41c5340c599baf7232992f771 P ad79851e8a374423b7967dbf87bdcb9b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "380fcbc24f8f4b78b16632f6a1c3687e" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 46491 } } peers { permanent_uuid: "ad79851e8a374423b7967dbf87bdcb9b" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 46579 } } peers { permanent_uuid: "130ff4307aa34e638e62a7e068c3595c" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 35461 } }
I20260810 11:32:43.591606 20475 leader_election.cc:288] T 551766b41c5340c599baf7232992f771 P ad79851e8a374423b7967dbf87bdcb9b [CANDIDATE]: Term 1 election: Requested vote from peers 380fcbc24f8f4b78b16632f6a1c3687e (127.8.18.195:46491), 130ff4307aa34e638e62a7e068c3595c (127.8.18.193:35461)
I20260810 11:32:43.592268 20115 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "551766b41c5340c599baf7232992f771" candidate_uuid: "ad79851e8a374423b7967dbf87bdcb9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "130ff4307aa34e638e62a7e068c3595c"
I20260810 11:32:43.592336 20401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "551766b41c5340c599baf7232992f771" candidate_uuid: "ad79851e8a374423b7967dbf87bdcb9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "380fcbc24f8f4b78b16632f6a1c3687e"
I20260810 11:32:43.592674 20401 raft_consensus.cc:3037] T 551766b41c5340c599baf7232992f771 P 380fcbc24f8f4b78b16632f6a1c3687e [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:32:43.592584 20115 raft_consensus.cc:3037] T 551766b41c5340c599baf7232992f771 P 130ff4307aa34e638e62a7e068c3595c [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:32:43.597939 20115 raft_consensus.cc:2454] T 551766b41c5340c599baf7232992f771 P 130ff4307aa34e638e62a7e068c3595c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad79851e8a374423b7967dbf87bdcb9b in term 1.
I20260810 11:32:43.597954 20401 raft_consensus.cc:2454] T 551766b41c5340c599baf7232992f771 P 380fcbc24f8f4b78b16632f6a1c3687e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad79851e8a374423b7967dbf87bdcb9b in term 1.
I20260810 11:32:43.598701 20196 leader_election.cc:302] T 551766b41c5340c599baf7232992f771 P ad79851e8a374423b7967dbf87bdcb9b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 130ff4307aa34e638e62a7e068c3595c, ad79851e8a374423b7967dbf87bdcb9b; no voters:
I20260810 11:32:43.599145 20475 raft_consensus.cc:2781] T 551766b41c5340c599baf7232992f771 P ad79851e8a374423b7967dbf87bdcb9b [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:32:43.599834 20475 raft_consensus.cc:686] T 551766b41c5340c599baf7232992f771 P ad79851e8a374423b7967dbf87bdcb9b [term 1 LEADER]: Becoming Leader. State: Replica: ad79851e8a374423b7967dbf87bdcb9b, State: Running, Role: LEADER
I20260810 11:32:43.600525 20475 consensus_queue.cc:237] T 551766b41c5340c599baf7232992f771 P ad79851e8a374423b7967dbf87bdcb9b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "380fcbc24f8f4b78b16632f6a1c3687e" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 46491 } } peers { permanent_uuid: "ad79851e8a374423b7967dbf87bdcb9b" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 46579 } } peers { permanent_uuid: "130ff4307aa34e638e62a7e068c3595c" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 35461 } }
I20260810 11:32:43.606659 19940 catalog_manager.cc:5495] T 551766b41c5340c599baf7232992f771 P ad79851e8a374423b7967dbf87bdcb9b reported cstate change: term changed from 0 to 1, leader changed from <none> to ad79851e8a374423b7967dbf87bdcb9b (127.8.18.194). New cstate: current_term: 1 leader_uuid: "ad79851e8a374423b7967dbf87bdcb9b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "380fcbc24f8f4b78b16632f6a1c3687e" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 46491 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ad79851e8a374423b7967dbf87bdcb9b" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 46579 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "130ff4307aa34e638e62a7e068c3595c" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 35461 } health_report { overall_health: UNKNOWN } } }
I20260810 11:32:43.659461 20464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:32:43.652332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50810 (local address 127.8.18.194:46579)
0810 11:32:43.652648 (+ 316us) server_negotiation.cc:206] Beginning negotiation
0810 11:32:43.652653 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:32:43.652672 (+ 19us) server_negotiation.cc:407] Connection header received
0810 11:32:43.652776 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:32:43.652779 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:32:43.652862 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:32:43.653004 (+ 142us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 11:32:43.653506 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:32:43.654609 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:32:43.656559 (+ 1950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:32:43.657181 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:32:43.657303 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:32:43.658069 (+ 766us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 11:32:43.658632 (+ 563us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 11:32:43.658793 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0810 11:32:43.658954 (+ 161us) server_negotiation.cc:299] Negotiation successful
0810 11:32:43.659087 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
I20260810 11:32:43.671248 20401 raft_consensus.cc:1261] T 551766b41c5340c599baf7232992f771 P 380fcbc24f8f4b78b16632f6a1c3687e [term 1 FOLLOWER]: Refusing update from remote peer ad79851e8a374423b7967dbf87bdcb9b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 11:32:43.671365 20115 raft_consensus.cc:1261] T 551766b41c5340c599baf7232992f771 P 130ff4307aa34e638e62a7e068c3595c [term 1 FOLLOWER]: Refusing update from remote peer ad79851e8a374423b7967dbf87bdcb9b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 11:32:43.672850 20478 consensus_queue.cc:1035] T 551766b41c5340c599baf7232992f771 P ad79851e8a374423b7967dbf87bdcb9b [LEADER]: Connected to new peer: Peer: permanent_uuid: "380fcbc24f8f4b78b16632f6a1c3687e" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 46491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 11:32:43.673485 20475 consensus_queue.cc:1035] T 551766b41c5340c599baf7232992f771 P ad79851e8a374423b7967dbf87bdcb9b [LEADER]: Connected to new peer: Peer: permanent_uuid: "130ff4307aa34e638e62a7e068c3595c" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 35461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 11:32:43.692795 20483 mvcc.cc:205] Tried to move back new op lower bound from 7316936964782813184 to 7316936964512845824. Current Snapshot: MvccSnapshot[applied={T|T < 7316936964782813184}]
I20260810 11:32:43.717970 19939 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43476:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 11:32:43.718719 19939 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43476:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 11:32:43.726049 19939 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 02097e106d234d4781a5e662f93ba496: Sending DeleteTablet for 3 replicas of tablet 551766b41c5340c599baf7232992f771
I20260810 11:32:43.727845 20242 tablet_service.cc:1514] Processing DeleteTablet for tablet 551766b41c5340c599baf7232992f771 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:32:43 UTC) from {username='kudu', principal='kudu/127.8.18.254@KRBTEST.COM'} at 127.0.0.1:50804
I20260810 11:32:43.727957 20381 tablet_service.cc:1514] Processing DeleteTablet for tablet 551766b41c5340c599baf7232992f771 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:32:43 UTC) from {username='kudu', principal='kudu/127.8.18.254@KRBTEST.COM'} at 127.0.0.1:56744
I20260810 11:32:43.727973 20095 tablet_service.cc:1514] Processing DeleteTablet for tablet 551766b41c5340c599baf7232992f771 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:32:43 UTC) from {username='kudu', principal='kudu/127.8.18.254@KRBTEST.COM'} at 127.0.0.1:49702
I20260810 11:32:43.734983 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 20025
I20260810 11:32:43.741739 20493 tablet_replica.cc:330] T 551766b41c5340c599baf7232992f771 P 380fcbc24f8f4b78b16632f6a1c3687e: stopping tablet replica
I20260810 11:32:43.742405 20492 tablet_replica.cc:330] T 551766b41c5340c599baf7232992f771 P 130ff4307aa34e638e62a7e068c3595c: stopping tablet replica
I20260810 11:32:43.742434 20493 raft_consensus.cc:2229] T 551766b41c5340c599baf7232992f771 P 380fcbc24f8f4b78b16632f6a1c3687e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 11:32:43.743647 20493 raft_consensus.cc:2258] T 551766b41c5340c599baf7232992f771 P 380fcbc24f8f4b78b16632f6a1c3687e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:32:43.744489 20491 tablet_replica.cc:330] T 551766b41c5340c599baf7232992f771 P ad79851e8a374423b7967dbf87bdcb9b: stopping tablet replica
I20260810 11:32:43.744954 20491 raft_consensus.cc:2229] T 551766b41c5340c599baf7232992f771 P ad79851e8a374423b7967dbf87bdcb9b [term 1 LEADER]: Raft consensus shutting down.
I20260810 11:32:43.746061 20491 raft_consensus.cc:2258] T 551766b41c5340c599baf7232992f771 P ad79851e8a374423b7967dbf87bdcb9b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:32:43.746419 20493 ts_tablet_manager.cc:1905] T 551766b41c5340c599baf7232992f771 P 380fcbc24f8f4b78b16632f6a1c3687e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:32:43.748616 20491 ts_tablet_manager.cc:1905] T 551766b41c5340c599baf7232992f771 P ad79851e8a374423b7967dbf87bdcb9b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:32:43.743621 20492 raft_consensus.cc:2229] T 551766b41c5340c599baf7232992f771 P 130ff4307aa34e638e62a7e068c3595c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 11:32:43.751956 20492 raft_consensus.cc:2258] T 551766b41c5340c599baf7232992f771 P 130ff4307aa34e638e62a7e068c3595c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:32:43.755522 20492 ts_tablet_manager.cc:1905] T 551766b41c5340c599baf7232992f771 P 130ff4307aa34e638e62a7e068c3595c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:32:43.757778 20493 ts_tablet_manager.cc:1918] T 551766b41c5340c599baf7232992f771 P 380fcbc24f8f4b78b16632f6a1c3687e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
Aug 10 11:32:43 dist-test-slave-srgv krb5kdc[19455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361557, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.18.193@KRBTEST.COM
I20260810 11:32:43.758082 20493 log.cc:1196] T 551766b41c5340c599baf7232992f771 P 380fcbc24f8f4b78b16632f6a1c3687e: Deleting WAL directory at /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786361401922362-8267-0/minicluster-data/ts-2/wal/wals/551766b41c5340c599baf7232992f771
I20260810 11:32:43.758705 20493 ts_tablet_manager.cc:1939] T 551766b41c5340c599baf7232992f771 P 380fcbc24f8f4b78b16632f6a1c3687e: Deleting consensus metadata
I20260810 11:32:43.761193 19925 catalog_manager.cc:4841] TS 380fcbc24f8f4b78b16632f6a1c3687e (127.8.18.195:46491): tablet 551766b41c5340c599baf7232992f771 (table test-table [id=fed626aa9a36405e9e9e68421ba26faa]) successfully deleted
I20260810 11:32:43.765847 20491 ts_tablet_manager.cc:1918] T 551766b41c5340c599baf7232992f771 P ad79851e8a374423b7967dbf87bdcb9b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 11:32:43.766146 20491 log.cc:1196] T 551766b41c5340c599baf7232992f771 P ad79851e8a374423b7967dbf87bdcb9b: Deleting WAL directory at /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786361401922362-8267-0/minicluster-data/ts-1/wal/wals/551766b41c5340c599baf7232992f771
I20260810 11:32:43.766392 20492 ts_tablet_manager.cc:1918] T 551766b41c5340c599baf7232992f771 P 130ff4307aa34e638e62a7e068c3595c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 11:32:43.766673 20492 log.cc:1196] T 551766b41c5340c599baf7232992f771 P 130ff4307aa34e638e62a7e068c3595c: Deleting WAL directory at /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786361401922362-8267-0/minicluster-data/ts-0/wal/wals/551766b41c5340c599baf7232992f771
I20260810 11:32:43.767006 20491 ts_tablet_manager.cc:1939] T 551766b41c5340c599baf7232992f771 P ad79851e8a374423b7967dbf87bdcb9b: Deleting consensus metadata
I20260810 11:32:43.767519 20492 ts_tablet_manager.cc:1939] T 551766b41c5340c599baf7232992f771 P 130ff4307aa34e638e62a7e068c3595c: Deleting consensus metadata
I20260810 11:32:43.769652 19923 catalog_manager.cc:4841] TS ad79851e8a374423b7967dbf87bdcb9b (127.8.18.194:46579): tablet 551766b41c5340c599baf7232992f771 (table test-table [id=fed626aa9a36405e9e9e68421ba26faa]) successfully deleted
I20260810 11:32:43.770238 19926 catalog_manager.cc:4841] TS 130ff4307aa34e638e62a7e068c3595c (127.8.18.193:35461): tablet 551766b41c5340c599baf7232992f771 (table test-table [id=fed626aa9a36405e9e9e68421ba26faa]) successfully deleted
I20260810 11:32:43.771296 20466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:32:43.737132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49710 (local address 127.8.18.193:35461)
0810 11:32:43.737459 (+ 327us) server_negotiation.cc:206] Beginning negotiation
0810 11:32:43.737466 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:32:43.738368 (+ 902us) server_negotiation.cc:407] Connection header received
0810 11:32:43.738481 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:32:43.738490 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:32:43.738658 (+ 168us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:32:43.738785 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:32:43.743417 (+ 4632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:32:43.746155 (+ 2738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:32:43.748585 (+ 2430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:32:43.749410 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:32:43.749612 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:32:43.759812 (+ 10200us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:32:43.759839 (+ 27us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:32:43.759858 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:32:43.759921 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:32:43.765647 (+ 5726us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:32:43.767044 (+ 1397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:32:43.767055 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:32:43.767059 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:32:43.767275 (+ 216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:32:43.767841 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:32:43.767846 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:32:43.767849 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:32:43.768459 (+ 610us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:32:43.770143 (+ 1684us) server_negotiation.cc:1035] Waiting for connection context
0810 11:32:43.770482 (+ 339us) server_negotiation.cc:299] Negotiation successful
0810 11:32:43.770753 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115}
I20260810 11:32:43.772164 20157 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:32:43.902817 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 20025
I20260810 11:32:43.931878 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 20168
Aug 10 11:32:43 dist-test-slave-srgv krb5kdc[19455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361557, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.18.194@KRBTEST.COM
I20260810 11:32:43.957966 20464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:32:43.932782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50820 (local address 127.8.18.194:46579)
0810 11:32:43.932978 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0810 11:32:43.932983 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:32:43.933091 (+ 108us) server_negotiation.cc:407] Connection header received
0810 11:32:43.933269 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:32:43.933273 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:32:43.933361 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:32:43.933564 (+ 203us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:32:43.935493 (+ 1929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:32:43.936383 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:32:43.938603 (+ 2220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:32:43.939113 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:32:43.939245 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:32:43.945952 (+ 6707us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:32:43.945978 (+ 26us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:32:43.946025 (+ 47us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:32:43.946129 (+ 104us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:32:43.953040 (+ 6911us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:32:43.954310 (+ 1270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:32:43.954323 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:32:43.954328 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:32:43.954688 (+ 360us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:32:43.955312 (+ 624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:32:43.955318 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:32:43.955322 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:32:43.955858 (+ 536us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:32:43.956232 (+ 374us) server_negotiation.cc:1035] Waiting for connection context
0810 11:32:43.957148 (+ 916us) server_negotiation.cc:299] Negotiation successful
0810 11:32:43.957516 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260810 11:32:43.958762 20303 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:32:44.085151 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 20168
I20260810 11:32:44.107285 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 20314
Aug 10 11:32:44 dist-test-slave-srgv krb5kdc[19455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361557, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.18.195@KRBTEST.COM
I20260810 11:32:44.124518 20497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:32:44.108267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56756 (local address 127.8.18.195:46491)
0810 11:32:44.108817 (+ 550us) server_negotiation.cc:206] Beginning negotiation
0810 11:32:44.108824 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:32:44.108853 (+ 29us) server_negotiation.cc:407] Connection header received
0810 11:32:44.108913 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:32:44.108917 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:32:44.108980 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:32:44.109086 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:32:44.110646 (+ 1560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:32:44.111383 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:32:44.112713 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:32:44.113144 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:32:44.113267 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:32:44.118203 (+ 4936us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:32:44.118224 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:32:44.118240 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:32:44.118295 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:32:44.121302 (+ 3007us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:32:44.122428 (+ 1126us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:32:44.122437 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:32:44.122439 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:32:44.122596 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:32:44.122964 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:32:44.122967 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:32:44.122969 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:32:44.123373 (+ 404us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:32:44.123611 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0810 11:32:44.123889 (+ 278us) server_negotiation.cc:299] Negotiation successful
0810 11:32:44.124099 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":408,"thread_start_us":244,"threads_started":1}
I20260810 11:32:44.125248 20443 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:32:44.222685 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 20314
I20260810 11:32:44.242698 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 19907
I20260810 11:32:44.244026 19971 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:32:44.367213 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 19907
I20260810 11:32:44.393725 8267 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260810 11:32:44.740554 8267 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260810 11:32:44.740870 8267 mini_ranger.cc:66] Stopping Ranger...
I20260810 11:32:45.087563 8267 mini_ranger.cc:68] Stopped Ranger
2026-08-10 11:32:45.087 UTC [19485] LOG: received smart shutdown request
2026-08-10 11:32:45.092 UTC [19485] LOG: background worker "logical replication launcher" (PID 19494) exited with exit code 1
2026-08-10 11:32:45.092 UTC [19489] LOG: shutting down
2026-08-10 11:32:45.302 UTC [19485] LOG: database system is shut down
2026-08-10T11:32:45Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (56127 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 11:32:45 dist-test-slave-srgv krb5kdc[20509](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 11:32:45 dist-test-slave-srgv krb5kdc[20509](info): set up 2 sockets
Aug 10 11:32:45 dist-test-slave-srgv krb5kdc[20509](info): commencing operation
krb5kdc: starting...
W20260810 11:32:46.642577 8267 mini_kdc.cc:121] Time spent starting KDC: real 1.151s user 0.000s sys 0.033s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/krb5kdc/test-user.keytab.
Aug 10 11:32:46 dist-test-slave-srgv krb5kdc[20509](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361566, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T11:32:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T11:32:46Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.8.18.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.8.18.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.8.18.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/krb5kdc/rangeradmin_127.8.18.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.8.18.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/krb5kdc/rangeradmin_127.8.18.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.8.18.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.8.18.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.8.18.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/krb5kdc/rangerlookup_127.8.18.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.8.18.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/krb5kdc/rangerlookup_127.8.18.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.8.18.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.8.18.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.8.18.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/krb5kdc/HTTP_127.8.18.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.8.18.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/krb5kdc/HTTP_127.8.18.212@KRBTEST.COM.keytab.
I20260810 11:32:47.044443 8267 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-task_r3aKW/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/minicluster-data/postgres -l logfile start
2026-08-10 11:32:49.973 UTC [20539] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-10 11:32:49.973 UTC [20539] LOG: listening on IPv4 address "127.8.18.212", port 55237
2026-08-10 11:32:49.977 UTC [20539] LOG: listening on Unix socket "/tmp/.s.PGSQL.55237"
2026-08-10 11:32:50.002 UTC [20542] LOG: database system was shut down at 2026-08-10 11:32:48 UTC
2026-08-10 11:32:50.007 UTC [20539] LOG: database system is ready to accept connections
I20260810 11:32:51.008270 8267 mini_postgres.cc:96] Postgres bound to 55237
2026-08-10 11:32:51.030 UTC [20550] FATAL: database "slave" does not exist
127.8.18.212:55237 - accepting connections
I20260810 11:32:51.031008 8267 mini_ranger.cc:161] Starting Ranger...
I20260810 11:32:51.063107 8267 mini_ranger.cc:84] Created miniranger Postgres user
I20260810 11:32:51.663164 8267 mini_ranger.cc:87] Created ranger Postgres database
I20260810 11:32:51.663290 8267 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/minicluster-data/ranger-admin
2026-08-10 11:32:51,982 [I] DB FLAVOR :POSTGRES
2026-08-10 11:32:51,982 [I] --------- Verifying Ranger DB connection ---------
2026-08-10 11:32:51,982 [I] Checking connection..
2026-08-10 11:32:51,982 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_r3aKW/build/asan/bin/postgresql.jar:/tmp/dist-test-task_r3aKW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.18.212:55237/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-10 11:32:52,260 [I] Checking connection passed.
2026-08-10 11:32:52,261 [I] --------- Verifying version history table ---------
2026-08-10 11:32:52,261 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_r3aKW/build/asan/bin/postgresql.jar:/tmp/dist-test-task_r3aKW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.18.212:55237/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-10 11:32:52,541 [I] Table x_db_version_h does not exist in database ranger
2026-08-10 11:32:52,541 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_r3aKW/build/asan/bin/postgresql.jar:/tmp/dist-test-task_r3aKW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.18.212:55237/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-10 11:32:52,834 [I] Table x_db_version_h does not exist in database ranger
2026-08-10 11:32:52,834 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-10 11:32:52,834 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_r3aKW/build/asan/bin/postgresql.jar:/tmp/dist-test-task_r3aKW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.18.212:55237/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task_r3aKW/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-10 11:32:53.086 UTC [20616] WARNING: there is no transaction in progress
2026-08-10 11:32:53,094 [I] create_dbversion_catalog.sql file imported successfully
2026-08-10 11:32:53,094 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_r3aKW/build/asan/bin/postgresql.jar:/tmp/dist-test-task_r3aKW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.18.212:55237/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-10 11:32:53,385 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-10 11:32:53,385 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-10 11:32:53,385 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_r3aKW/build/asan/bin/postgresql.jar:/tmp/dist-test-task_r3aKW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.18.212:55237/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-10 11:32:53,675 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_r3aKW/build/asan/bin/postgresql.jar:/tmp/dist-test-task_r3aKW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.18.212:55237/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-10 11:32:53,973 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_r3aKW/build/asan/bin/postgresql.jar:/tmp/dist-test-task_r3aKW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.18.212:55237/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-srgv.c.gcp-upstream.internal','N') ;"
2026-08-10 11:32:54,231 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_r3aKW/build/asan/bin/postgresql.jar:/tmp/dist-test-task_r3aKW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.18.212:55237/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-10 11:32:54,514 [I] Table x_portal_user does not exist in database ranger
2026-08-10 11:32:54,514 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_r3aKW/build/asan/bin/postgresql.jar:/tmp/dist-test-task_r3aKW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.18.212:55237/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-10 11:32:54,779 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-10 11:32:54,779 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_r3aKW/build/asan/bin/postgresql.jar:/tmp/dist-test-task_r3aKW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.18.212:55237/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-10 11:32:55,038 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_r3aKW/build/asan/bin/postgresql.jar:/tmp/dist-test-task_r3aKW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.18.212:55237/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-10 11:32:55,304 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-10 11:32:55,305 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_r3aKW/build/asan/bin/postgresql.jar:/tmp/dist-test-task_r3aKW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.18.212:55237/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task_r3aKW/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-10 11:32:56.478 UTC [20727] WARNING: there is no transaction in progress
2026-08-10 11:32:56.499 UTC [20727] WARNING: there is no transaction in progress
2026-08-10 11:32:56.521 UTC [20727] WARNING: there is no transaction in progress
2026-08-10 11:32:56.541 UTC [20727] WARNING: there is no transaction in progress
2026-08-10 11:32:56.561 UTC [20727] WARNING: there is no transaction in progress
2026-08-10 11:32:56.583 UTC [20727] WARNING: there is no transaction in progress
2026-08-10 11:32:56.648 UTC [20727] WARNING: there is no transaction in progress
2026-08-10 11:32:56.656 UTC [20727] WARNING: there is no transaction in progress
2026-08-10 11:32:56.676 UTC [20727] WARNING: there is no transaction in progress
2026-08-10 11:32:56.689 UTC [20727] WARNING: there is no transaction in progress
2026-08-10 11:32:56.702 UTC [20727] WARNING: there is no transaction in progress
2026-08-10 11:32:56.718 UTC [20727] WARNING: there is no transaction in progress
2026-08-10 11:32:56.729 UTC [20727] WARNING: there is no transaction in progress
2026-08-10 11:32:56.737 UTC [20727] WARNING: there is no transaction in progress
2026-08-10 11:32:56.746 UTC [20727] WARNING: there is no transaction in progress
2026-08-10 11:32:56.754 UTC [20727] WARNING: there is no transaction in progress
2026-08-10 11:32:57,624 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-10 11:32:57,624 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_r3aKW/build/asan/bin/postgresql.jar:/tmp/dist-test-task_r3aKW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.18.212:55237/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-10 11:32:57,880 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-10 11:32:57,881 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_r3aKW/build/asan/bin/postgresql.jar:/tmp/dist-test-task_r3aKW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.18.212:55237/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-10 11:32:58,155 [I] Table x_portal_user already exists in database 'ranger'
2026-08-10 11:32:58,155 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_r3aKW/build/asan/bin/postgresql.jar:/tmp/dist-test-task_r3aKW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.18.212:55237/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-10 11:32:58,417 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-10 11:32:58,418 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_r3aKW/build/asan/bin/postgresql.jar:/tmp/dist-test-task_r3aKW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.18.212:55237/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-10 11:32:58,672 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_r3aKW/build/asan/bin/postgresql.jar:/tmp/dist-test-task_r3aKW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.18.212:55237/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-10 11:32:58,933 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_r3aKW/build/asan/bin/postgresql.jar:/tmp/dist-test-task_r3aKW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.18.212:55237/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-srgv.c.gcp-upstream.internal';"
2026-08-10 11:32:59,209 [I] CORE_DB_SCHEMA import status has been updated
2026-08-10 11:32:59,210 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_r3aKW/build/asan/bin/postgresql.jar:/tmp/dist-test-task_r3aKW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.18.212:55237/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-10 11:32:59,508 [I] DB_PATCHES have already been applied
I20260810 11:32:59.515215 8267 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/minicluster-data/ranger-admin:/tmp/dist-test-task_r3aKW/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-task_r3aKW/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-task_r3aKW/build/asan/bin/postgresql.jar:/tmp/dist-test-task_r3aKW/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260810 11:32:59.515312 8267 mini_ranger.cc:193] Using host: 127.8.18.212
I20260810 11:32:59.534672 8267 mini_ranger.cc:227] Ranger admin URL: http://127.8.18.212:55767
Aug 10, 2026 11:32:59 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 10, 2026 11:32:59 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 10, 2026 11:33:00 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/minicluster-data/ranger-admin/ews/webapp
Aug 10, 2026 11:33:00 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 10, 2026 11:33:00 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 10, 2026 11:33:00 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/minicluster-data/ranger-admin/ews/webapp].
Aug 10, 2026 11:33:00 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260810 11:33:00.733880 8267 mini_ranger.cc:160] Time spent starting Ranger: real 9.703s user 0.000s sys 0.057s
Aug 10, 2026 11:33:24 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 10, 2026 11:33:25 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 10, 2026 11:33:25 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 10, 2026 11:33:25 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 10, 2026 11:33:25 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 10, 2026 11:33:25 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 10, 2026 11:33:25 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 10, 2026 11:33:25 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 10, 2026 11:33:25 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 10, 2026 11:33:25 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 10, 2026 11:33:25 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 10, 2026 11:33:25 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 10, 2026 11:33:25 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 10, 2026 11:33:25 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 10, 2026 11:33:25 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 10, 2026 11:33:25 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 10, 2026 11:33:25 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 10, 2026 11:33:26 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260810 11:33:26.886348 8267 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.8.18.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.8.18.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.8.18.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/krb5kdc/rangerkms_127.8.18.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.8.18.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/krb5kdc/rangerkms_127.8.18.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.8.18.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.8.18.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.8.18.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/krb5kdc/HTTP_127.8.18.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.8.18.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/krb5kdc/HTTP_127.8.18.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 10 11:33:27 dist-test-slave-srgv krb5kdc[20509](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361607, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260810 11:33:27.092826 8267 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260810 11:33:27.121254 8267 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260810 11:33:28.188118 8267 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260810 11:33:28.188277 8267 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/minicluster-data/ranger-kms
I20260810 11:33:28.188364 8267 mini_ranger_kms.cc:227] Using postgres at 127.8.18.212:55237
2026-08-10 11:33:28,604 [I] DB FLAVOR :POSTGRES
2026-08-10 11:33:28,605 [I] --------- Verifying Ranger DB connection ---------
2026-08-10 11:33:28,605 [I] Checking connection
2026-08-10 11:33:28,605 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_r3aKW/build/asan/bin/postgresql.jar:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.18.212:55237/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-10 11:33:28,880 [I] connection success
2026-08-10 11:33:28,880 [I] --------- Verifying Ranger DB tables ---------
2026-08-10 11:33:28,880 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-10 11:33:28,880 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_r3aKW/build/asan/bin/postgresql.jar:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.18.212:55237/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-10 11:33:29,155 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-10 11:33:29,155 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-10 11:33:29,156 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-10 11:33:29,156 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task_r3aKW/build/asan/bin/postgresql.jar:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.18.212:55237/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-10 11:33:29,453 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260810 11:33:30.059589 8267 mini_ranger_kms.cc:310] Created kms service
I20260810 11:33:30.277031 8267 mini_ranger_kms.cc:326] Created kudu user
I20260810 11:33:30.333478 8267 mini_ranger_kms.cc:343] Created rangerkms user
Aug 10 11:33:30 dist-test-slave-srgv krb5kdc[20509](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361607, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.8.18.212@KRBTEST.COM
I20260810 11:33:30.856598 8267 mini_ranger_kms.cc:384] Added ranger policy
I20260810 11:33:30.857014 8267 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/minicluster-data/ranger-kms:/tmp/dist-test-task_r3aKW/build/asan/bin/postgresql.jar:/tmp/dist-test-task_r3aKW/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-task_r3aKW/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260810 11:33:30.857080 8267 mini_ranger_kms.cc:240] Using host: 127.8.18.212
I20260810 11:33:30.878755 8267 mini_ranger_kms.cc:276] Ranger KMS PID: 20910
I20260810 11:33:30.878921 8267 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.8.18.212:41093
Aug 10, 2026 11:33:31 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 10, 2026 11:33:32 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 10, 2026 11:33:32 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 10, 2026 11:33:32 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 10, 2026 11:33:32 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/minicluster-data/ranger-kms/ews/webapp].
Aug 10, 2026 11:33:32 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260810 11:33:33.238665 8267 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.146s user 0.104s sys 0.339s
I20260810 11:33:33.238965 8267 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260810 11:33:33.239174 8267 mini_ranger_kms.cc:401] 127.8.18.212:41093/kms/v1/keys
Aug 10 11:33:33 dist-test-slave-srgv krb5kdc[20509](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786361613, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.8.18.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 10 11:33:33 dist-test-slave-srgv krb5kdc[20509](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361613, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.8.18.212@KRBTEST.COM for HTTP/127.8.18.212@KRBTEST.COM
Aug 10 11:33:34 dist-test-slave-srgv krb5kdc[20509](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361613, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.8.18.212@KRBTEST.COM for HTTP/127.8.18.212@KRBTEST.COM
Aug 10 11:33:36 dist-test-slave-srgv krb5kdc[20509](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361616, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.8.18.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.18.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.18.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.254.keytab.
Entry for principal kudu/127.8.18.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.254.keytab.
I20260810 11:33:36.898985 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.18.254:36507
--webserver_interface=127.8.18.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.254.keytab
--principal=kudu/127.8.18.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.18.212:33245
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.18.254:36507
--ranger_config_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.18.212:41093/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:33:37.272219 20961 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:33:37.272634 20961 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:33:37.272840 20961 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:33:37.280793 20961 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 11:33:37.280995 20961 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:33:37.281121 20961 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:33:37.281232 20961 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 11:33:37.281328 20961 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 11:33:37.292824 20961 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.18.212:33245
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/minicluster-data/master-0/wal
--ranger_kms_url=127.8.18.212:41093/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.8.18.254:36507
--ranger_config_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.254.keytab
--principal=kudu/127.8.18.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.18.254:36507
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.18.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:33:37.294464 20961 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:33:37.296026 20961 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:33:37.310400 20966 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:33:37.310482 20969 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:33:37.310411 20967 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:33:37.311354 20961 server_base.cc:1034] running on GCE node
I20260810 11:33:37.312285 20961 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:33:37.314167 20961 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:33:37.317690 20961 hybrid_clock.cc:648] HybridClock initialized: now 1786361617317535 us; error 98 us; skew 500 ppm
I20260810 11:33:37.318274 20961 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:33:37 dist-test-slave-srgv krb5kdc[20509](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361617, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:33:37.328250 20961 init.cc:377] Logged in from keytab as kudu/127.8.18.254@KRBTEST.COM (short username kudu)
I20260810 11:33:37.330426 20961 webserver.cc:460] Webserver started at http://127.8.18.254:46031/ using document root <none> and password file <none>
I20260810 11:33:37.331305 20961 fs_manager.cc:359] Metadata directory not provided
I20260810 11:33:37.331439 20961 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:33:37.331753 20961 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 11:33:37 dist-test-slave-srgv krb5kdc[20509](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361617, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.254@KRBTEST.COM for HTTP/127.8.18.212@KRBTEST.COM
I20260810 11:33:37.389366 20961 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/minicluster-data/master-0/data/instance:
uuid: "d482e868edcf44a3a887277ae45225c8"
format_stamp: "Formatted at 2026-08-10 11:33:37 on dist-test-slave-srgv"
server_key: "c9c6f513049fd63e42519ca356cfead8"
server_key_iv: "445373a43a18f77134f65079ff38b2ad"
server_key_version: "kuduclusterkey@0"
I20260810 11:33:37.390618 20961 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/minicluster-data/master-0/wal/instance:
uuid: "d482e868edcf44a3a887277ae45225c8"
format_stamp: "Formatted at 2026-08-10 11:33:37 on dist-test-slave-srgv"
server_key: "c9c6f513049fd63e42519ca356cfead8"
server_key_iv: "445373a43a18f77134f65079ff38b2ad"
server_key_version: "kuduclusterkey@0"
I20260810 11:33:37.397600 20961 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.002s
I20260810 11:33:37.418063 20976 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:33:37.420337 20961 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.000s sys 0.004s
I20260810 11:33:37.420607 20961 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/minicluster-data/master-0/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/minicluster-data/master-0/wal
uuid: "d482e868edcf44a3a887277ae45225c8"
format_stamp: "Formatted at 2026-08-10 11:33:37 on dist-test-slave-srgv"
server_key: "c9c6f513049fd63e42519ca356cfead8"
server_key_iv: "445373a43a18f77134f65079ff38b2ad"
server_key_version: "kuduclusterkey@0"
I20260810 11:33:37.420899 20961 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:33:37.458994 20961 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:33:37.464371 20961 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:33:37.464804 20961 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:33:37.487278 20961 rpc_server.cc:307] RPC server started. Bound to: 127.8.18.254:36507
I20260810 11:33:37.487327 21027 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.18.254:36507 every 8 connection(s)
I20260810 11:33:37.488909 20961 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
I20260810 11:33:37.494560 21028 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:33:37.494995 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 20961
I20260810 11:33:37.495267 8267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/minicluster-data/master-0/wal/instance
I20260810 11:33:37.496272 8267 external_mini_cluster.cc:1428] Setting key e3ecdf392eb5fc14687bb6897ce5c0f2
I20260810 11:33:37.512303 21028 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d482e868edcf44a3a887277ae45225c8: Bootstrap starting.
Aug 10 11:33:37 dist-test-slave-srgv krb5kdc[20509](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361616, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.18.254@KRBTEST.COM
I20260810 11:33:37.515990 21028 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d482e868edcf44a3a887277ae45225c8: Neither blocks nor log segments found. Creating new log.
I20260810 11:33:37.517529 21028 log.cc:824] T 00000000000000000000000000000000 P d482e868edcf44a3a887277ae45225c8: Log is configured to *not* fsync() on all Append() calls
I20260810 11:33:37.522269 21028 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d482e868edcf44a3a887277ae45225c8: No bootstrap required, opened a new log
I20260810 11:33:37.527626 21031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:33:37.498891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55330 (local address 127.8.18.254:36507)
0810 11:33:37.499863 (+ 972us) server_negotiation.cc:206] Beginning negotiation
0810 11:33:37.499876 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 11:33:37.499927 (+ 51us) server_negotiation.cc:407] Connection header received
0810 11:33:37.503416 (+ 3489us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:33:37.503459 (+ 43us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:33:37.504181 (+ 722us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:33:37.504783 (+ 602us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:33:37.506435 (+ 1652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:33:37.508557 (+ 2122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:33:37.510440 (+ 1883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:33:37.511577 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:33:37.511801 (+ 224us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:33:37.516310 (+ 4509us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:33:37.516323 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:33:37.516347 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:33:37.516412 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:33:37.521680 (+ 5268us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:33:37.522737 (+ 1057us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:33:37.522747 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:33:37.522757 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:33:37.523038 (+ 281us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:33:37.523585 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:33:37.523590 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:33:37.523593 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:33:37.524523 (+ 930us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:33:37.524894 (+ 371us) server_negotiation.cc:1035] Waiting for connection context
0810 11:33:37.525363 (+ 469us) server_negotiation.cc:299] Negotiation successful
0810 11:33:37.525933 (+ 570us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":722,"thread_start_us":472,"threads_started":1}
I20260810 11:33:37.533764 21028 raft_consensus.cc:348] T 00000000000000000000000000000000 P d482e868edcf44a3a887277ae45225c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d482e868edcf44a3a887277ae45225c8" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 36507 } }
I20260810 11:33:37.534289 21028 raft_consensus.cc:374] T 00000000000000000000000000000000 P d482e868edcf44a3a887277ae45225c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:33:37.534457 21028 raft_consensus.cc:729] T 00000000000000000000000000000000 P d482e868edcf44a3a887277ae45225c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d482e868edcf44a3a887277ae45225c8, State: Initialized, Role: FOLLOWER
I20260810 11:33:37.535118 21028 consensus_queue.cc:260] T 00000000000000000000000000000000 P d482e868edcf44a3a887277ae45225c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d482e868edcf44a3a887277ae45225c8" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 36507 } }
I20260810 11:33:37.535375 21028 raft_consensus.cc:388] T 00000000000000000000000000000000 P d482e868edcf44a3a887277ae45225c8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 11:33:37.535496 21028 raft_consensus.cc:482] T 00000000000000000000000000000000 P d482e868edcf44a3a887277ae45225c8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 11:33:37.535647 21028 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d482e868edcf44a3a887277ae45225c8 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:33:37.538610 21028 raft_consensus.cc:504] T 00000000000000000000000000000000 P d482e868edcf44a3a887277ae45225c8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d482e868edcf44a3a887277ae45225c8" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 36507 } }
I20260810 11:33:37.539232 21028 leader_election.cc:302] T 00000000000000000000000000000000 P d482e868edcf44a3a887277ae45225c8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d482e868edcf44a3a887277ae45225c8; no voters:
I20260810 11:33:37.539927 21028 leader_election.cc:288] T 00000000000000000000000000000000 P d482e868edcf44a3a887277ae45225c8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 11:33:37.540086 21033 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d482e868edcf44a3a887277ae45225c8 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:33:37.541465 21033 raft_consensus.cc:686] T 00000000000000000000000000000000 P d482e868edcf44a3a887277ae45225c8 [term 1 LEADER]: Becoming Leader. State: Replica: d482e868edcf44a3a887277ae45225c8, State: Running, Role: LEADER
I20260810 11:33:37.542196 21033 consensus_queue.cc:237] T 00000000000000000000000000000000 P d482e868edcf44a3a887277ae45225c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d482e868edcf44a3a887277ae45225c8" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 36507 } }
I20260810 11:33:37.542785 21028 sys_catalog.cc:564] T 00000000000000000000000000000000 P d482e868edcf44a3a887277ae45225c8 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 11:33:37.545888 21034 sys_catalog.cc:455] T 00000000000000000000000000000000 P d482e868edcf44a3a887277ae45225c8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d482e868edcf44a3a887277ae45225c8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d482e868edcf44a3a887277ae45225c8" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 36507 } } }
I20260810 11:33:37.547106 21034 sys_catalog.cc:458] T 00000000000000000000000000000000 P d482e868edcf44a3a887277ae45225c8 [sys.catalog]: This master's current role is: LEADER
I20260810 11:33:37.549546 21035 sys_catalog.cc:455] T 00000000000000000000000000000000 P d482e868edcf44a3a887277ae45225c8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d482e868edcf44a3a887277ae45225c8. Latest consensus state: current_term: 1 leader_uuid: "d482e868edcf44a3a887277ae45225c8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d482e868edcf44a3a887277ae45225c8" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 36507 } } }
I20260810 11:33:37.550616 21035 sys_catalog.cc:458] T 00000000000000000000000000000000 P d482e868edcf44a3a887277ae45225c8 [sys.catalog]: This master's current role is: LEADER
I20260810 11:33:37.551757 21042 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 11:33:37.558751 21042 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 11:33:37.559598 21028 ranger_client.cc:318] Using new properties file: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260810 11:33:37.586006 21042 catalog_manager.cc:1347] Generated new cluster ID: 341c6a56e85a4ff7a8531e4e8dbee7ce
I20260810 11:33:37.586248 21042 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 11:33:37.640782 21042 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 11:33:37.643886 21042 catalog_manager.cc:1504] Loading token signing keys...
I20260810 11:33:37.654753 21042 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d482e868edcf44a3a887277ae45225c8: Generated new TSK 0
I20260810 11:33:37.656219 21042 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.18.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.18.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.18.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.193.keytab.
Entry for principal kudu/127.8.18.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.193.keytab.
I20260810 11:33:38.462747 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.18.212:41093/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.18.193:0
--local_ip_for_outbound_sockets=127.8.18.193
--webserver_interface=127.8.18.193
--webserver_port=0
--tserver_master_addrs=127.8.18.254:36507
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.193.keytab
--principal=kudu/127.8.18.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.18.212:33245
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 11:33:38 dist-test-slave-srgv krb5kdc[20509](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786361618, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260810 11:33:38.941193 21073 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:33:38.941751 21073 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:33:38.941959 21073 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:33:38.954449 21073 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:33:38.954705 21073 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:33:38.954933 21073 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.18.193
I20260810 11:33:38.968691 21073 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.18.212:33245
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.8.18.212:41093/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.193.keytab
--principal=kudu/127.8.18.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.18.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.18.193
--webserver_port=0
--tserver_master_addrs=127.8.18.254:36507
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.18.193
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:33:38.970326 21073 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:33:38.971997 21073 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:33:38.988020 21084 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:33:38.987995 21083 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:33:38.988778 21086 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:33:38.989482 21073 server_base.cc:1034] running on GCE node
I20260810 11:33:38.990307 21073 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:33:38.991590 21073 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:33:39.001574 21073 hybrid_clock.cc:648] HybridClock initialized: now 1786361618997455 us; error 4093 us; skew 500 ppm
I20260810 11:33:39.002092 21073 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:33:39 dist-test-slave-srgv krb5kdc[20509](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361619, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:33:39.006826 21073 init.cc:377] Logged in from keytab as kudu/127.8.18.193@KRBTEST.COM (short username kudu)
I20260810 11:33:39.009025 21073 webserver.cc:460] Webserver started at http://127.8.18.193:44073/ using document root <none> and password file <none>
I20260810 11:33:39.009951 21073 fs_manager.cc:359] Metadata directory not provided
I20260810 11:33:39.010113 21073 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:33:39.010455 21073 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 11:33:39 dist-test-slave-srgv krb5kdc[20509](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361619, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.193@KRBTEST.COM for HTTP/127.8.18.212@KRBTEST.COM
I20260810 11:33:39.029708 21073 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/minicluster-data/ts-0/data/instance:
uuid: "fb558a810efd415cb0bf25c474b3bf99"
format_stamp: "Formatted at 2026-08-10 11:33:39 on dist-test-slave-srgv"
server_key: "20525b6b54ee8c5f4fe5c5f60cbbcaa7"
server_key_iv: "9c442cd2354b6f0023b1c55cf99dd660"
server_key_version: "kuduclusterkey@0"
I20260810 11:33:39.031093 21073 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/minicluster-data/ts-0/wal/instance:
uuid: "fb558a810efd415cb0bf25c474b3bf99"
format_stamp: "Formatted at 2026-08-10 11:33:39 on dist-test-slave-srgv"
server_key: "20525b6b54ee8c5f4fe5c5f60cbbcaa7"
server_key_iv: "9c442cd2354b6f0023b1c55cf99dd660"
server_key_version: "kuduclusterkey@0"
I20260810 11:33:39.046020 21073 fs_manager.cc:691] Time spent creating directory manager: real 0.014s user 0.009s sys 0.004s
I20260810 11:33:39.086239 21094 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:33:39.088950 21073 fs_manager.cc:725] Time spent opening block manager: real 0.018s user 0.007s sys 0.005s
I20260810 11:33:39.089293 21073 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/minicluster-data/ts-0/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/minicluster-data/ts-0/wal
uuid: "fb558a810efd415cb0bf25c474b3bf99"
format_stamp: "Formatted at 2026-08-10 11:33:39 on dist-test-slave-srgv"
server_key: "20525b6b54ee8c5f4fe5c5f60cbbcaa7"
server_key_iv: "9c442cd2354b6f0023b1c55cf99dd660"
server_key_version: "kuduclusterkey@0"
I20260810 11:33:39.089737 21073 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:33:39.174037 21073 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:33:39.181346 21073 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:33:39.181850 21073 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:33:39.193915 21073 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
Aug 10 11:33:39 dist-test-slave-srgv krb5kdc[20509](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361618, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.254@KRBTEST.COM for HTTP/127.8.18.212@KRBTEST.COM
I20260810 11:33:39.208168 21073 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:33:39.208338 21073 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260810 11:33:39.208506 21073 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:33:39.208648 21073 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:33:39.301098 21073 rpc_server.cc:307] RPC server started. Bound to: 127.8.18.193:45081
I20260810 11:33:39.301151 21208 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.18.193:45081 every 8 connection(s)
I20260810 11:33:39.303032 21073 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/minicluster-data/ts-0/data/info.pb
I20260810 11:33:39.305241 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 21073
I20260810 11:33:39.305518 8267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/minicluster-data/ts-0/wal/instance
I20260810 11:33:39.306365 8267 external_mini_cluster.cc:1428] Setting key 0a7871417ec4a67565cfefdc2691e08d
Aug 10 11:33:39 dist-test-slave-srgv krb5kdc[20509](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361619, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.193@KRBTEST.COM for kudu/127.8.18.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.8.18.194@KRBTEST.COM; defaulting to no policy
I20260810 11:33:39.360232 21212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:33:39.319393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.193:34461 (local address 127.8.18.254:36507)
0810 11:33:39.334661 (+ 15268us) server_negotiation.cc:206] Beginning negotiation
0810 11:33:39.334669 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:33:39.334698 (+ 29us) server_negotiation.cc:407] Connection header received
0810 11:33:39.337389 (+ 2691us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:33:39.337395 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:33:39.337549 (+ 154us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:33:39.337700 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:33:39.341623 (+ 3923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:33:39.342890 (+ 1267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:33:39.345922 (+ 3032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:33:39.346588 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:33:39.346734 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:33:39.352049 (+ 5315us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:33:39.352058 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:33:39.352063 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:33:39.352123 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:33:39.356014 (+ 3891us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:33:39.357176 (+ 1162us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:33:39.357183 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:33:39.357187 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:33:39.357358 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:33:39.357982 (+ 624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:33:39.357987 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:33:39.357991 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:33:39.358372 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:33:39.358508 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0810 11:33:39.359562 (+ 1054us) server_negotiation.cc:299] Negotiation successful
0810 11:33:39.359827 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15126,"thread_start_us":15293,"threads_started":1}
I20260810 11:33:39.362787 21213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:33:39.320082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:36507 (local address 127.8.18.193:34461)
0810 11:33:39.334230 (+ 14148us) negotiation.cc:107] Waiting for socket to connect
0810 11:33:39.334298 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0810 11:33:39.336859 (+ 2561us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:33:39.338662 (+ 1803us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:33:39.338686 (+ 24us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:33:39.339389 (+ 703us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:33:39.340308 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:33:39.340337 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:33:39.343202 (+ 2865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:33:39.343209 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:33:39.345612 (+ 2403us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:33:39.345645 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:33:39.347110 (+ 1465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:33:39.347117 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:33:39.347461 (+ 344us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:33:39.348388 (+ 927us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:33:39.348421 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:33:39.351698 (+ 3277us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:33:39.356277 (+ 4579us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:33:39.356288 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:33:39.356309 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:33:39.357000 (+ 691us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:33:39.357542 (+ 542us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:33:39.357547 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:33:39.357551 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:33:39.357839 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:33:39.358597 (+ 758us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:33:39.358607 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:33:39.359106 (+ 499us) client_negotiation.cc:769] Sending connection context
0810 11:33:39.360562 (+ 1456us) client_negotiation.cc:240] Negotiation successful
0810 11:33:39.361282 (+ 720us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13741,"thread_start_us":17205,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.18.194@KRBTEST.COM" created.
I20260810 11:33:39.365329 21209 heartbeater.cc:347] Connected to a master server at 127.8.18.254:36507
I20260810 11:33:39.365736 21209 heartbeater.cc:464] Registering TS with master...
I20260810 11:33:39.366742 21209 heartbeater.cc:511] Master 127.8.18.254:36507 requested a full tablet report, sending...
I20260810 11:33:39.369856 20993 ts_manager.cc:194] Registered new tserver with Master: fb558a810efd415cb0bf25c474b3bf99 (127.8.18.193:45081)
I20260810 11:33:39.372193 20993 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.18.193@KRBTEST.COM'} at 127.8.18.193:34461
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.18.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.194.keytab.
Entry for principal kudu/127.8.18.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.194.keytab.
I20260810 11:33:39.402199 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.18.212:41093/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.18.194:0
--local_ip_for_outbound_sockets=127.8.18.194
--webserver_interface=127.8.18.194
--webserver_port=0
--tserver_master_addrs=127.8.18.254:36507
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.194.keytab
--principal=kudu/127.8.18.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.18.212:33245
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 11:33:39 dist-test-slave-srgv krb5kdc[20509](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361618, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.254@KRBTEST.COM for HTTP/127.8.18.212@KRBTEST.COM
W20260810 11:33:39.898159 21215 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:33:39.898536 21215 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:33:39.898674 21215 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:33:39.907477 21215 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:33:39.907644 21215 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:33:39.907773 21215 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.18.194
I20260810 11:33:39.919373 21215 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.18.212:33245
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.8.18.212:41093/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.194.keytab
--principal=kudu/127.8.18.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.18.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.18.194
--webserver_port=0
--tserver_master_addrs=127.8.18.254:36507
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.18.194
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:33:39.920909 21215 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:33:39.922441 21215 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:33:39.937099 21232 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:33:39.937222 21230 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:33:39.937219 21229 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:33:39.937985 21215 server_base.cc:1034] running on GCE node
I20260810 11:33:39.938688 21215 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:33:39.939836 21215 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:33:39.941126 21215 hybrid_clock.cc:648] HybridClock initialized: now 1786361619941062 us; error 44 us; skew 500 ppm
I20260810 11:33:39.941591 21215 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:33:39 dist-test-slave-srgv krb5kdc[20509](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361619, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:33:39.946463 21215 init.cc:377] Logged in from keytab as kudu/127.8.18.194@KRBTEST.COM (short username kudu)
I20260810 11:33:39.950084 21215 webserver.cc:460] Webserver started at http://127.8.18.194:37795/ using document root <none> and password file <none>
I20260810 11:33:39.951154 21215 fs_manager.cc:359] Metadata directory not provided
I20260810 11:33:39.951337 21215 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:33:39.951794 21215 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 11:33:39 dist-test-slave-srgv krb5kdc[20509](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361619, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.194@KRBTEST.COM for HTTP/127.8.18.212@KRBTEST.COM
I20260810 11:33:39.970438 21215 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/minicluster-data/ts-1/data/instance:
uuid: "94767f4c56104676b69351a3d83c31f9"
format_stamp: "Formatted at 2026-08-10 11:33:39 on dist-test-slave-srgv"
server_key: "1a0065482ff0b396f47b2b0712ad780e"
server_key_iv: "1a864ea2b5ae37df07e2383f800c7c5e"
server_key_version: "kuduclusterkey@0"
I20260810 11:33:39.971561 21215 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/minicluster-data/ts-1/wal/instance:
uuid: "94767f4c56104676b69351a3d83c31f9"
format_stamp: "Formatted at 2026-08-10 11:33:39 on dist-test-slave-srgv"
server_key: "1a0065482ff0b396f47b2b0712ad780e"
server_key_iv: "1a864ea2b5ae37df07e2383f800c7c5e"
server_key_version: "kuduclusterkey@0"
I20260810 11:33:39.978246 21215 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260810 11:33:39.995602 21239 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:33:39.997783 21215 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260810 11:33:39.998032 21215 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/minicluster-data/ts-1/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/minicluster-data/ts-1/wal
uuid: "94767f4c56104676b69351a3d83c31f9"
format_stamp: "Formatted at 2026-08-10 11:33:39 on dist-test-slave-srgv"
server_key: "1a0065482ff0b396f47b2b0712ad780e"
server_key_iv: "1a864ea2b5ae37df07e2383f800c7c5e"
server_key_version: "kuduclusterkey@0"
I20260810 11:33:39.998317 21215 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:33:40.029973 21215 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:33:40.035214 21215 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:33:40.035583 21215 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:33:40.036705 21215 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:33:40.038508 21215 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:33:40.038630 21215 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:33:40.038805 21215 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:33:40.038889 21215 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:33:40.084981 21215 rpc_server.cc:307] RPC server started. Bound to: 127.8.18.194:44583
I20260810 11:33:40.084993 21352 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.18.194:44583 every 8 connection(s)
I20260810 11:33:40.086589 21215 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/minicluster-data/ts-1/data/info.pb
I20260810 11:33:40.089244 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 21215
I20260810 11:33:40.089469 8267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/minicluster-data/ts-1/wal/instance
I20260810 11:33:40.090127 8267 external_mini_cluster.cc:1428] Setting key 302a4f6205da99bcde51012d38875224
Aug 10 11:33:40 dist-test-slave-srgv krb5kdc[20509](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361619, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.194@KRBTEST.COM for kudu/127.8.18.254@KRBTEST.COM
I20260810 11:33:40.118270 21355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:33:40.089700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.194:36721 (local address 127.8.18.254:36507)
0810 11:33:40.090215 (+ 515us) server_negotiation.cc:206] Beginning negotiation
0810 11:33:40.090222 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:33:40.092189 (+ 1967us) server_negotiation.cc:407] Connection header received
0810 11:33:40.094266 (+ 2077us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:33:40.094275 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:33:40.094394 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:33:40.094524 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:33:40.096851 (+ 2327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:33:40.098139 (+ 1288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:33:40.100767 (+ 2628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:33:40.101602 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:33:40.101769 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:33:40.111392 (+ 9623us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:33:40.111399 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:33:40.111403 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:33:40.111447 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:33:40.114609 (+ 3162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:33:40.115557 (+ 948us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:33:40.115565 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:33:40.115574 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:33:40.115719 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:33:40.116254 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:33:40.116260 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:33:40.116266 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:33:40.116586 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:33:40.116747 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0810 11:33:40.117584 (+ 837us) server_negotiation.cc:299] Negotiation successful
0810 11:33:40.117857 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":384,"thread_start_us":248,"threads_started":1}
I20260810 11:33:40.119555 21356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:33:40.090467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:36507 (local address 127.8.18.194:36721)
0810 11:33:40.091431 (+ 964us) negotiation.cc:107] Waiting for socket to connect
0810 11:33:40.091486 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0810 11:33:40.093855 (+ 2369us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:33:40.094898 (+ 1043us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:33:40.094911 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:33:40.095599 (+ 688us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:33:40.096606 (+ 1007us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:33:40.096634 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:33:40.098356 (+ 1722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:33:40.098361 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:33:40.100508 (+ 2147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:33:40.100540 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:33:40.101903 (+ 1363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:33:40.101912 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:33:40.102397 (+ 485us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:33:40.103373 (+ 976us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:33:40.103406 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:33:40.111032 (+ 7626us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:33:40.114822 (+ 3790us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:33:40.114834 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:33:40.114861 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:33:40.115399 (+ 538us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:33:40.115865 (+ 466us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:33:40.115870 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:33:40.115874 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:33:40.116124 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:33:40.116751 (+ 627us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:33:40.116763 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:33:40.117145 (+ 382us) client_negotiation.cc:769] Sending connection context
0810 11:33:40.117488 (+ 343us) client_negotiation.cc:240] Negotiation successful
0810 11:33:40.118093 (+ 605us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":641,"thread_start_us":415,"threads_started":1}
I20260810 11:33:40.121246 21353 heartbeater.cc:347] Connected to a master server at 127.8.18.254:36507
I20260810 11:33:40.121661 21353 heartbeater.cc:464] Registering TS with master...
I20260810 11:33:40.122493 21353 heartbeater.cc:511] Master 127.8.18.254:36507 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.8.18.195@KRBTEST.COM; defaulting to no policy
I20260810 11:33:40.124161 20993 ts_manager.cc:194] Registered new tserver with Master: 94767f4c56104676b69351a3d83c31f9 (127.8.18.194:44583)
I20260810 11:33:40.125051 20993 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.18.194@KRBTEST.COM'} at 127.8.18.194:36721
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.18.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.18.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.195.keytab.
Entry for principal kudu/127.8.18.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.195.keytab.
I20260810 11:33:40.170681 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.18.212:41093/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.18.195:0
--local_ip_for_outbound_sockets=127.8.18.195
--webserver_interface=127.8.18.195
--webserver_port=0
--tserver_master_addrs=127.8.18.254:36507
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.195.keytab
--principal=kudu/127.8.18.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.18.212:33245
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 11:33:40.377014 21209 heartbeater.cc:503] Master 127.8.18.254:36507 was elected leader, sending a full tablet report...
W20260810 11:33:40.558892 21359 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:33:40.559232 21359 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:33:40.559405 21359 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:33:40.567673 21359 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:33:40.567824 21359 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:33:40.568019 21359 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.18.195
I20260810 11:33:40.579294 21359 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.18.212:33245
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.8.18.212:41093/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.195.keytab
--principal=kudu/127.8.18.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.18.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.18.195
--webserver_port=0
--tserver_master_addrs=127.8.18.254:36507
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.18.195
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:33:40.580850 21359 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:33:40.582468 21359 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:33:40.597815 21365 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:33:40.597797 21367 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:33:40.597925 21364 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:33:40.603420 21359 server_base.cc:1034] running on GCE node
I20260810 11:33:40.604319 21359 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:33:40.605556 21359 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:33:40.606798 21359 hybrid_clock.cc:648] HybridClock initialized: now 1786361620606751 us; error 41 us; skew 500 ppm
I20260810 11:33:40.607191 21359 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:33:40 dist-test-slave-srgv krb5kdc[20509](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361620, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:33:40.611063 21359 init.cc:377] Logged in from keytab as kudu/127.8.18.195@KRBTEST.COM (short username kudu)
I20260810 11:33:40.612882 21359 webserver.cc:460] Webserver started at http://127.8.18.195:34233/ using document root <none> and password file <none>
I20260810 11:33:40.613835 21359 fs_manager.cc:359] Metadata directory not provided
I20260810 11:33:40.614006 21359 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:33:40.614362 21359 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 11:33:40 dist-test-slave-srgv krb5kdc[20509](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361620, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.195@KRBTEST.COM for HTTP/127.8.18.212@KRBTEST.COM
I20260810 11:33:40.633126 21359 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/minicluster-data/ts-2/data/instance:
uuid: "584fbc0fd7f0407a9ea8811cf932d743"
format_stamp: "Formatted at 2026-08-10 11:33:40 on dist-test-slave-srgv"
server_key: "8b67f1fe9319dbdcb5ca43fb33f2280a"
server_key_iv: "42f2d30358c6663180eeed516a6db927"
server_key_version: "kuduclusterkey@0"
I20260810 11:33:40.634632 21359 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/minicluster-data/ts-2/wal/instance:
uuid: "584fbc0fd7f0407a9ea8811cf932d743"
format_stamp: "Formatted at 2026-08-10 11:33:40 on dist-test-slave-srgv"
server_key: "8b67f1fe9319dbdcb5ca43fb33f2280a"
server_key_iv: "42f2d30358c6663180eeed516a6db927"
server_key_version: "kuduclusterkey@0"
I20260810 11:33:40.640815 21359 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260810 11:33:40.660729 21374 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:33:40.662747 21359 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260810 11:33:40.662986 21359 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/minicluster-data/ts-2/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/minicluster-data/ts-2/wal
uuid: "584fbc0fd7f0407a9ea8811cf932d743"
format_stamp: "Formatted at 2026-08-10 11:33:40 on dist-test-slave-srgv"
server_key: "8b67f1fe9319dbdcb5ca43fb33f2280a"
server_key_iv: "42f2d30358c6663180eeed516a6db927"
server_key_version: "kuduclusterkey@0"
I20260810 11:33:40.663241 21359 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:33:40.683226 21359 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:33:40.688427 21359 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:33:40.688843 21359 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:33:40.690089 21359 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:33:40.691754 21359 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:33:40.691843 21359 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:33:40.692013 21359 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:33:40.692075 21359 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:33:40.730297 21359 rpc_server.cc:307] RPC server started. Bound to: 127.8.18.195:44519
I20260810 11:33:40.730336 21488 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.18.195:44519 every 8 connection(s)
I20260810 11:33:40.731739 21359 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/minicluster-data/ts-2/data/info.pb
I20260810 11:33:40.737219 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 21359
I20260810 11:33:40.737380 8267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/minicluster-data/ts-2/wal/instance
I20260810 11:33:40.737912 8267 external_mini_cluster.cc:1428] Setting key a14ddbd4b933f1f69fe069d119d80220
Aug 10 11:33:40 dist-test-slave-srgv krb5kdc[20509](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361620, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.195@KRBTEST.COM for kudu/127.8.18.254@KRBTEST.COM
I20260810 11:33:40.755609 21491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:33:40.734753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.195:59367 (local address 127.8.18.254:36507)
0810 11:33:40.735285 (+ 532us) server_negotiation.cc:206] Beginning negotiation
0810 11:33:40.735292 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:33:40.736389 (+ 1097us) server_negotiation.cc:407] Connection header received
0810 11:33:40.738552 (+ 2163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:33:40.738557 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:33:40.738636 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:33:40.738742 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:33:40.740643 (+ 1901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:33:40.741600 (+ 957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:33:40.743437 (+ 1837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:33:40.744273 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:33:40.744410 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:33:40.748263 (+ 3853us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:33:40.748270 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:33:40.748273 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:33:40.748313 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:33:40.751661 (+ 3348us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:33:40.752811 (+ 1150us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:33:40.752815 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:33:40.752818 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:33:40.753035 (+ 217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:33:40.753650 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:33:40.753660 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:33:40.753662 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:33:40.754071 (+ 409us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:33:40.754243 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0810 11:33:40.755088 (+ 845us) server_negotiation.cc:299] Negotiation successful
0810 11:33:40.755325 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":383,"thread_start_us":247,"threads_started":1}
I20260810 11:33:40.756718 21492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:33:40.735351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:36507 (local address 127.8.18.195:59367)
0810 11:33:40.736190 (+ 839us) negotiation.cc:107] Waiting for socket to connect
0810 11:33:40.736238 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0810 11:33:40.738186 (+ 1948us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:33:40.739036 (+ 850us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:33:40.739047 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:33:40.739637 (+ 590us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:33:40.740426 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:33:40.740448 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:33:40.741773 (+ 1325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:33:40.741777 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:33:40.743238 (+ 1461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:33:40.743260 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:33:40.744517 (+ 1257us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:33:40.744527 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:33:40.744852 (+ 325us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:33:40.745645 (+ 793us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:33:40.745689 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:33:40.747944 (+ 2255us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:33:40.751882 (+ 3938us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:33:40.751893 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:33:40.751911 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:33:40.752634 (+ 723us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:33:40.753171 (+ 537us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:33:40.753175 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:33:40.753177 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:33:40.753480 (+ 303us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:33:40.754251 (+ 771us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:33:40.754259 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:33:40.754688 (+ 429us) client_negotiation.cc:769] Sending connection context
0810 11:33:40.754945 (+ 257us) client_negotiation.cc:240] Negotiation successful
0810 11:33:40.755439 (+ 494us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":550,"thread_start_us":347,"threads_started":1}
I20260810 11:33:40.758603 21489 heartbeater.cc:347] Connected to a master server at 127.8.18.254:36507
I20260810 11:33:40.759055 21489 heartbeater.cc:464] Registering TS with master...
I20260810 11:33:40.759946 21489 heartbeater.cc:511] Master 127.8.18.254:36507 requested a full tablet report, sending...
I20260810 11:33:40.762053 20993 ts_manager.cc:194] Registered new tserver with Master: 584fbc0fd7f0407a9ea8811cf932d743 (127.8.18.195:44519)
I20260810 11:33:40.762915 20993 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.18.195@KRBTEST.COM'} at 127.8.18.195:59367
I20260810 11:33:40.768392 8267 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 11:33:40.770263 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 21073
Aug 10 11:33:40 dist-test-slave-srgv krb5kdc[20509](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361616, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.18.193@KRBTEST.COM
I20260810 11:33:40.791370 21494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:33:40.771487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45276 (local address 127.8.18.193:45081)
0810 11:33:40.772292 (+ 805us) server_negotiation.cc:206] Beginning negotiation
0810 11:33:40.772299 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:33:40.772328 (+ 29us) server_negotiation.cc:407] Connection header received
0810 11:33:40.772394 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:33:40.772400 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:33:40.772611 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:33:40.772762 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:33:40.774939 (+ 2177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:33:40.776125 (+ 1186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:33:40.777997 (+ 1872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:33:40.778692 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:33:40.778813 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:33:40.783901 (+ 5088us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:33:40.783922 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:33:40.783939 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:33:40.783986 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:33:40.787788 (+ 3802us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:33:40.788840 (+ 1052us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:33:40.788850 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:33:40.788853 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:33:40.789049 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:33:40.789596 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:33:40.789600 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:33:40.789603 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:33:40.789995 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:33:40.790296 (+ 301us) server_negotiation.cc:1035] Waiting for connection context
0810 11:33:40.790596 (+ 300us) server_negotiation.cc:299] Negotiation successful
0810 11:33:40.790922 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":639,"thread_start_us":448,"threads_started":1}
I20260810 11:33:40.792804 21204 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:33:40.885730 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 21073
I20260810 11:33:40.905320 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 21215
Aug 10 11:33:40 dist-test-slave-srgv krb5kdc[20509](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361616, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.18.194@KRBTEST.COM
I20260810 11:33:40.931388 21496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:33:40.906747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59346 (local address 127.8.18.194:44583)
0810 11:33:40.907370 (+ 623us) server_negotiation.cc:206] Beginning negotiation
0810 11:33:40.907377 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:33:40.907409 (+ 32us) server_negotiation.cc:407] Connection header received
0810 11:33:40.907471 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:33:40.907477 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:33:40.907713 (+ 236us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:33:40.907851 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:33:40.909491 (+ 1640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:33:40.910586 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:33:40.912245 (+ 1659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:33:40.912876 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:33:40.913155 (+ 279us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:33:40.924147 (+ 10992us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:33:40.924167 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:33:40.924193 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:33:40.924250 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:33:40.927749 (+ 3499us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:33:40.928730 (+ 981us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:33:40.928739 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:33:40.928742 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:33:40.928915 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:33:40.929387 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:33:40.929393 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:33:40.929396 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:33:40.929885 (+ 489us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:33:40.930291 (+ 406us) server_negotiation.cc:1035] Waiting for connection context
0810 11:33:40.930622 (+ 331us) server_negotiation.cc:299] Negotiation successful
0810 11:33:40.930946 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":459,"thread_start_us":324,"threads_started":1}
I20260810 11:33:40.933028 21348 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:33:41.022572 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 21215
I20260810 11:33:41.040568 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 21359
Aug 10 11:33:41 dist-test-slave-srgv krb5kdc[20509](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361616, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.18.195@KRBTEST.COM
I20260810 11:33:41.060742 21498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:33:41.041648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51878 (local address 127.8.18.195:44519)
0810 11:33:41.042454 (+ 806us) server_negotiation.cc:206] Beginning negotiation
0810 11:33:41.042462 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:33:41.042496 (+ 34us) server_negotiation.cc:407] Connection header received
0810 11:33:41.042630 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:33:41.042644 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:33:41.042904 (+ 260us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:33:41.043058 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:33:41.044701 (+ 1643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:33:41.046013 (+ 1312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:33:41.047469 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:33:41.048113 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:33:41.048282 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:33:41.053622 (+ 5340us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:33:41.053639 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:33:41.053656 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:33:41.053712 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:33:41.057139 (+ 3427us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:33:41.058229 (+ 1090us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:33:41.058238 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:33:41.058241 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:33:41.058407 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:33:41.058927 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:33:41.058931 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:33:41.058933 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:33:41.059363 (+ 430us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:33:41.059643 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0810 11:33:41.060014 (+ 371us) server_negotiation.cc:299] Negotiation successful
0810 11:33:41.060337 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":606,"thread_start_us":411,"threads_started":1}
I20260810 11:33:41.062285 21484 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:33:41.148890 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 21359
I20260810 11:33:41.166524 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 20961
I20260810 11:33:41.168042 21023 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:33:41.279958 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 20961
I20260810 11:33:41.306366 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.18.254:36507
--webserver_interface=127.8.18.254
--webserver_port=46031
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.254.keytab
--principal=kudu/127.8.18.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.18.212:33245
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.18.254:36507
--ranger_config_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.18.212:41093/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.8.18.212:41093/kms with env {KRB5CCNAME=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:33:41.684072 21501 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:33:41.684543 21501 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:33:41.684724 21501 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:33:41.692727 21501 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 11:33:41.692875 21501 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:33:41.692925 21501 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:33:41.692988 21501 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 11:33:41.693040 21501 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 11:33:41.704361 21501 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.18.212:33245
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.8.18.212:41093/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.8.18.254:36507
--ranger_config_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.254.keytab
--principal=kudu/127.8.18.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.18.254:36507
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.18.254
--webserver_port=46031
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:33:41.705963 21501 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:33:41.707872 21501 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:33:41.721167 21507 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:33:41.721268 21506 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:33:41.721268 21509 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:33:41.722576 21501 server_base.cc:1034] running on GCE node
I20260810 11:33:41.723579 21501 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:33:41.725294 21501 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:33:41.726576 21501 hybrid_clock.cc:648] HybridClock initialized: now 1786361621726523 us; error 55 us; skew 500 ppm
I20260810 11:33:41.726977 21501 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:33:41 dist-test-slave-srgv krb5kdc[20509](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361621, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:33:41.730803 21501 init.cc:377] Logged in from keytab as kudu/127.8.18.254@KRBTEST.COM (short username kudu)
I20260810 11:33:41.732959 21501 webserver.cc:460] Webserver started at http://127.8.18.254:46031/ using document root <none> and password file <none>
I20260810 11:33:41.733919 21501 fs_manager.cc:359] Metadata directory not provided
I20260810 11:33:41.734097 21501 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 10 11:33:41 dist-test-slave-srgv krb5kdc[20509](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361621, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.254@KRBTEST.COM for HTTP/127.8.18.212@KRBTEST.COM
I20260810 11:33:41.825176 21501 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.000s sys 0.004s
I20260810 11:33:41.829210 21517 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:33:41.831171 21501 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260810 11:33:41.831445 21501 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/minicluster-data/master-0/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/minicluster-data/master-0/wal
uuid: "d482e868edcf44a3a887277ae45225c8"
format_stamp: "Formatted at 2026-08-10 11:33:37 on dist-test-slave-srgv"
server_key: "c9c6f513049fd63e42519ca356cfead8"
server_key_iv: "445373a43a18f77134f65079ff38b2ad"
server_key_version: "kuduclusterkey@0"
I20260810 11:33:41.832653 21501 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:33:41.856359 21501 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:33:41.861205 21501 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:33:41.861579 21501 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:33:41.882946 21501 rpc_server.cc:307] RPC server started. Bound to: 127.8.18.254:36507
I20260810 11:33:41.882985 21569 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.18.254:36507 every 8 connection(s)
I20260810 11:33:41.884631 21501 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
I20260810 11:33:41.890105 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 21501
I20260810 11:33:41.891362 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.18.212:41093/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.18.193:45081
--local_ip_for_outbound_sockets=127.8.18.193
--tserver_master_addrs=127.8.18.254:36507
--webserver_port=44073
--webserver_interface=127.8.18.193
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.193.keytab
--principal=kudu/127.8.18.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.18.212:33245
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.8.18.212:41093/kms with env {KRB5CCNAME=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 11:33:41.893910 21570 sys_catalog.cc:263] Verifying existing consensus state
I20260810 11:33:41.899781 21570 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d482e868edcf44a3a887277ae45225c8: Bootstrap starting.
I20260810 11:33:41.920882 21570 log.cc:824] T 00000000000000000000000000000000 P d482e868edcf44a3a887277ae45225c8: Log is configured to *not* fsync() on all Append() calls
I20260810 11:33:41.931358 21570 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d482e868edcf44a3a887277ae45225c8: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 11:33:41.932001 21570 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d482e868edcf44a3a887277ae45225c8: Bootstrap complete.
I20260810 11:33:41.942586 21570 raft_consensus.cc:348] T 00000000000000000000000000000000 P d482e868edcf44a3a887277ae45225c8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d482e868edcf44a3a887277ae45225c8" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 36507 } }
I20260810 11:33:41.943714 21570 raft_consensus.cc:729] T 00000000000000000000000000000000 P d482e868edcf44a3a887277ae45225c8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d482e868edcf44a3a887277ae45225c8, State: Initialized, Role: FOLLOWER
I20260810 11:33:41.944689 21570 consensus_queue.cc:260] T 00000000000000000000000000000000 P d482e868edcf44a3a887277ae45225c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d482e868edcf44a3a887277ae45225c8" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 36507 } }
I20260810 11:33:41.945051 21570 raft_consensus.cc:388] T 00000000000000000000000000000000 P d482e868edcf44a3a887277ae45225c8 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 11:33:41.945182 21570 raft_consensus.cc:482] T 00000000000000000000000000000000 P d482e868edcf44a3a887277ae45225c8 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 11:33:41.945384 21570 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d482e868edcf44a3a887277ae45225c8 [term 1 FOLLOWER]: Advancing to term 2
I20260810 11:33:41.952529 21570 raft_consensus.cc:504] T 00000000000000000000000000000000 P d482e868edcf44a3a887277ae45225c8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d482e868edcf44a3a887277ae45225c8" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 36507 } }
I20260810 11:33:41.953277 21570 leader_election.cc:302] T 00000000000000000000000000000000 P d482e868edcf44a3a887277ae45225c8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d482e868edcf44a3a887277ae45225c8; no voters:
I20260810 11:33:41.956400 21570 leader_election.cc:288] T 00000000000000000000000000000000 P d482e868edcf44a3a887277ae45225c8 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260810 11:33:41.956547 21574 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d482e868edcf44a3a887277ae45225c8 [term 2 FOLLOWER]: Leader election won for term 2
I20260810 11:33:41.959631 21570 sys_catalog.cc:564] T 00000000000000000000000000000000 P d482e868edcf44a3a887277ae45225c8 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 11:33:41.961812 21574 raft_consensus.cc:686] T 00000000000000000000000000000000 P d482e868edcf44a3a887277ae45225c8 [term 2 LEADER]: Becoming Leader. State: Replica: d482e868edcf44a3a887277ae45225c8, State: Running, Role: LEADER
I20260810 11:33:41.962795 21574 consensus_queue.cc:237] T 00000000000000000000000000000000 P d482e868edcf44a3a887277ae45225c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d482e868edcf44a3a887277ae45225c8" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 36507 } }
I20260810 11:33:41.966483 21575 sys_catalog.cc:455] T 00000000000000000000000000000000 P d482e868edcf44a3a887277ae45225c8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "d482e868edcf44a3a887277ae45225c8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d482e868edcf44a3a887277ae45225c8" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 36507 } } }
I20260810 11:33:41.967104 21575 sys_catalog.cc:458] T 00000000000000000000000000000000 P d482e868edcf44a3a887277ae45225c8 [sys.catalog]: This master's current role is: LEADER
I20260810 11:33:41.967967 21576 sys_catalog.cc:455] T 00000000000000000000000000000000 P d482e868edcf44a3a887277ae45225c8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d482e868edcf44a3a887277ae45225c8. Latest consensus state: current_term: 2 leader_uuid: "d482e868edcf44a3a887277ae45225c8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d482e868edcf44a3a887277ae45225c8" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 36507 } } }
I20260810 11:33:41.968397 21576 sys_catalog.cc:458] T 00000000000000000000000000000000 P d482e868edcf44a3a887277ae45225c8 [sys.catalog]: This master's current role is: LEADER
I20260810 11:33:41.973577 21584 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 11:33:41.981503 21584 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 11:33:41.983604 21584 catalog_manager.cc:1259] Loaded cluster ID: 341c6a56e85a4ff7a8531e4e8dbee7ce
I20260810 11:33:41.983768 21584 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 11:33:41.986702 21570 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260810 11:33:41.988816 21584 catalog_manager.cc:1504] Loading token signing keys...
I20260810 11:33:41.991135 21584 catalog_manager.cc:5879] T 00000000000000000000000000000000 P d482e868edcf44a3a887277ae45225c8: Loaded TSK: 0
I20260810 11:33:41.993507 21584 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260810 11:33:42.380642 21572 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:33:42.381125 21572 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:33:42.381320 21572 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:33:42.394631 21572 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:33:42.394842 21572 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:33:42.395066 21572 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.18.193
I20260810 11:33:42.408761 21572 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.18.212:33245
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.8.18.212:41093/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.193.keytab
--principal=kudu/127.8.18.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.18.193:45081
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.18.193
--webserver_port=44073
--tserver_master_addrs=127.8.18.254:36507
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.18.193
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:33:42.410384 21572 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:33:42.411913 21572 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:33:42.446139 21607 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:33:42.449146 21608 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:33:42.459412 21610 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:33:42.460731 21572 server_base.cc:1034] running on GCE node
I20260810 11:33:42.461948 21572 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:33:42.464561 21572 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:33:42.472671 21572 hybrid_clock.cc:648] HybridClock initialized: now 1786361622472539 us; error 101 us; skew 500 ppm
I20260810 11:33:42.473439 21572 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:33:42 dist-test-slave-srgv krb5kdc[20509](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361622, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:33:42.481645 21572 init.cc:377] Logged in from keytab as kudu/127.8.18.193@KRBTEST.COM (short username kudu)
I20260810 11:33:42.485358 21572 webserver.cc:460] Webserver started at http://127.8.18.193:44073/ using document root <none> and password file <none>
I20260810 11:33:42.486757 21572 fs_manager.cc:359] Metadata directory not provided
I20260810 11:33:42.486999 21572 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 10 11:33:42 dist-test-slave-srgv krb5kdc[20509](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361622, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.193@KRBTEST.COM for HTTP/127.8.18.212@KRBTEST.COM
I20260810 11:33:42.538079 21572 fs_manager.cc:709] Time spent opening directory manager: real 0.019s user 0.006s sys 0.005s
I20260810 11:33:42.557895 21618 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:33:42.560541 21572 fs_manager.cc:725] Time spent opening block manager: real 0.020s user 0.008s sys 0.012s
I20260810 11:33:42.560945 21572 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/minicluster-data/ts-0/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/minicluster-data/ts-0/wal
uuid: "fb558a810efd415cb0bf25c474b3bf99"
format_stamp: "Formatted at 2026-08-10 11:33:39 on dist-test-slave-srgv"
server_key: "20525b6b54ee8c5f4fe5c5f60cbbcaa7"
server_key_iv: "9c442cd2354b6f0023b1c55cf99dd660"
server_key_version: "kuduclusterkey@0"
I20260810 11:33:42.562830 21572 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:33:42.588824 21572 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:33:42.594259 21572 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:33:42.594604 21572 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:33:42.598163 21572 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:33:42.600143 21572 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:33:42.600306 21572 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:33:42.600461 21572 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:33:42.600565 21572 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:33:42.681269 21572 rpc_server.cc:307] RPC server started. Bound to: 127.8.18.193:45081
I20260810 11:33:42.681355 21733 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.18.193:45081 every 8 connection(s)
I20260810 11:33:42.683233 21572 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/minicluster-data/ts-0/data/info.pb
I20260810 11:33:42.689553 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 21572
I20260810 11:33:42.690732 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.18.212:41093/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.18.194:44583
--local_ip_for_outbound_sockets=127.8.18.194
--tserver_master_addrs=127.8.18.254:36507
--webserver_port=37795
--webserver_interface=127.8.18.194
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.194.keytab
--principal=kudu/127.8.18.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.18.212:33245
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.8.18.212:41093/kms with env {KRB5CCNAME=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 11:33:42 dist-test-slave-srgv krb5kdc[20509](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361622, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.193@KRBTEST.COM for kudu/127.8.18.254@KRBTEST.COM
I20260810 11:33:42.732795 21737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:33:42.687626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:36507 (local address 127.8.18.193:60153)
0810 11:33:42.699738 (+ 12112us) negotiation.cc:107] Waiting for socket to connect
0810 11:33:42.699823 (+ 85us) client_negotiation.cc:174] Beginning negotiation
0810 11:33:42.702918 (+ 3095us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:33:42.705500 (+ 2582us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:33:42.705516 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:33:42.706316 (+ 800us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:33:42.707515 (+ 1199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:33:42.707552 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:33:42.710839 (+ 3287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:33:42.710849 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:33:42.713593 (+ 2744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:33:42.713635 (+ 42us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:33:42.715828 (+ 2193us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:33:42.715843 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:33:42.716205 (+ 362us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:33:42.717297 (+ 1092us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:33:42.717338 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:33:42.720807 (+ 3469us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:33:42.726932 (+ 6125us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:33:42.726947 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:33:42.726971 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:33:42.727789 (+ 818us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:33:42.728485 (+ 696us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:33:42.728491 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:33:42.728495 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:33:42.728776 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:33:42.729862 (+ 1086us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:33:42.729872 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:33:42.730310 (+ 438us) client_negotiation.cc:769] Sending connection context
0810 11:33:42.730607 (+ 297us) client_negotiation.cc:240] Negotiation successful
0810 11:33:42.731277 (+ 670us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11713,"thread_start_us":11508,"threads_started":1}
I20260810 11:33:42.734658 21736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:33:42.687570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.193:60153 (local address 127.8.18.254:36507)
0810 11:33:42.701336 (+ 13766us) server_negotiation.cc:206] Beginning negotiation
0810 11:33:42.701365 (+ 29us) server_negotiation.cc:399] Waiting for connection header
0810 11:33:42.701477 (+ 112us) server_negotiation.cc:407] Connection header received
0810 11:33:42.703915 (+ 2438us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:33:42.703938 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:33:42.704618 (+ 680us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:33:42.705161 (+ 543us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:33:42.707921 (+ 2760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:33:42.710442 (+ 2521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:33:42.713944 (+ 3502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:33:42.715416 (+ 1472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:33:42.715722 (+ 306us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:33:42.721321 (+ 5599us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:33:42.721337 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:33:42.721366 (+ 29us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:33:42.721473 (+ 107us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:33:42.726632 (+ 5159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:33:42.728060 (+ 1428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:33:42.728071 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:33:42.728080 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:33:42.728331 (+ 251us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:33:42.728925 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:33:42.728930 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:33:42.728934 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:33:42.729692 (+ 758us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:33:42.732068 (+ 2376us) server_negotiation.cc:1035] Waiting for connection context
0810 11:33:42.732463 (+ 395us) server_negotiation.cc:299] Negotiation successful
0810 11:33:42.732979 (+ 516us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12843,"thread_start_us":11085,"threads_started":1}
I20260810 11:33:42.739279 21734 heartbeater.cc:347] Connected to a master server at 127.8.18.254:36507
I20260810 11:33:42.739763 21734 heartbeater.cc:464] Registering TS with master...
I20260810 11:33:42.740886 21734 heartbeater.cc:511] Master 127.8.18.254:36507 requested a full tablet report, sending...
W20260810 11:33:42.745033 21734 heartbeater.cc:644] Failed to heartbeat to 127.8.18.254:36507 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260810 11:33:42.745328 21734 heartbeater.cc:464] Registering TS with master...
I20260810 11:33:42.745682 21734 heartbeater.cc:511] Master 127.8.18.254:36507 requested a full tablet report, sending...
I20260810 11:33:42.747304 21734 heartbeater.cc:464] Registering TS with master...
I20260810 11:33:42.747727 21734 heartbeater.cc:511] Master 127.8.18.254:36507 requested a full tablet report, sending...
W20260810 11:33:42.749095 21734 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260810 11:33:43.175220 21738 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:33:43.175611 21738 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:33:43.175793 21738 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:33:43.184580 21738 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:33:43.184782 21738 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:33:43.184962 21738 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.18.194
I20260810 11:33:43.197216 21738 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.18.212:33245
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.8.18.212:41093/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.194.keytab
--principal=kudu/127.8.18.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.18.194:44583
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.18.194
--webserver_port=37795
--tserver_master_addrs=127.8.18.254:36507
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.18.194
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:33:43.198846 21738 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:33:43.200264 21738 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:33:43.218170 21752 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:33:43.218199 21753 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:33:43.219244 21738 server_base.cc:1034] running on GCE node
W20260810 11:33:43.219501 21755 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:33:43.220273 21738 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:33:43.221488 21738 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:33:43.222745 21738 hybrid_clock.cc:648] HybridClock initialized: now 1786361623222699 us; error 49 us; skew 500 ppm
I20260810 11:33:43.223116 21738 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:33:43 dist-test-slave-srgv krb5kdc[20509](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361623, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:33:43.226825 21738 init.cc:377] Logged in from keytab as kudu/127.8.18.194@KRBTEST.COM (short username kudu)
I20260810 11:33:43.230994 21738 webserver.cc:460] Webserver started at http://127.8.18.194:37795/ using document root <none> and password file <none>
I20260810 11:33:43.231951 21738 fs_manager.cc:359] Metadata directory not provided
I20260810 11:33:43.232152 21738 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 10 11:33:43 dist-test-slave-srgv krb5kdc[20509](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361623, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.194@KRBTEST.COM for HTTP/127.8.18.212@KRBTEST.COM
I20260810 11:33:43.261610 21738 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.002s sys 0.004s
I20260810 11:33:43.272326 21763 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:33:43.279786 21738 fs_manager.cc:725] Time spent opening block manager: real 0.016s user 0.003s sys 0.008s
I20260810 11:33:43.280172 21738 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/minicluster-data/ts-1/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/minicluster-data/ts-1/wal
uuid: "94767f4c56104676b69351a3d83c31f9"
format_stamp: "Formatted at 2026-08-10 11:33:39 on dist-test-slave-srgv"
server_key: "1a0065482ff0b396f47b2b0712ad780e"
server_key_iv: "1a864ea2b5ae37df07e2383f800c7c5e"
server_key_version: "kuduclusterkey@0"
I20260810 11:33:43.281900 21738 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:33:43.327963 21738 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:33:43.333726 21738 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:33:43.334105 21738 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:33:43.335489 21738 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:33:43.337779 21738 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:33:43.337965 21738 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:33:43.338124 21738 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:33:43.338241 21738 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 11:33:43 dist-test-slave-srgv krb5kdc[20509](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786361623, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:33:43.407884 21738 rpc_server.cc:307] RPC server started. Bound to: 127.8.18.194:44583
I20260810 11:33:43.407966 21878 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.18.194:44583 every 8 connection(s)
I20260810 11:33:43.410097 21738 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/minicluster-data/ts-1/data/info.pb
I20260810 11:33:43.414685 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 21738
I20260810 11:33:43.415800 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.18.212:41093/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.18.195:44519
--local_ip_for_outbound_sockets=127.8.18.195
--tserver_master_addrs=127.8.18.254:36507
--webserver_port=34233
--webserver_interface=127.8.18.195
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.195.keytab
--principal=kudu/127.8.18.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.18.212:33245
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.8.18.212:41093/kms with env {KRB5CCNAME=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 11:33:43 dist-test-slave-srgv krb5kdc[20509](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361623, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.194@KRBTEST.COM for kudu/127.8.18.254@KRBTEST.COM
I20260810 11:33:43.455852 21882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:33:43.418556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.194:53199 (local address 127.8.18.254:36507)
0810 11:33:43.430267 (+ 11711us) server_negotiation.cc:206] Beginning negotiation
0810 11:33:43.430276 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 11:33:43.430310 (+ 34us) server_negotiation.cc:407] Connection header received
0810 11:33:43.432955 (+ 2645us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:33:43.432964 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:33:43.433098 (+ 134us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:33:43.433220 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:33:43.437201 (+ 3981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:33:43.438511 (+ 1310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:33:43.441323 (+ 2812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:33:43.442088 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:33:43.442236 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:33:43.447337 (+ 5101us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:33:43.447345 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:33:43.447350 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:33:43.447400 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:33:43.451402 (+ 4002us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:33:43.452626 (+ 1224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:33:43.452637 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:33:43.452643 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:33:43.452874 (+ 231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:33:43.453546 (+ 672us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:33:43.453553 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:33:43.453557 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:33:43.453993 (+ 436us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:33:43.454166 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0810 11:33:43.455158 (+ 992us) server_negotiation.cc:299] Negotiation successful
0810 11:33:43.455441 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11527,"thread_start_us":11775,"threads_started":1}
I20260810 11:33:43.457377 21881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:33:43.417971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:36507 (local address 127.8.18.194:53199)
0810 11:33:43.420645 (+ 2674us) negotiation.cc:107] Waiting for socket to connect
0810 11:33:43.420722 (+ 77us) client_negotiation.cc:174] Beginning negotiation
0810 11:33:43.432267 (+ 11545us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:33:43.434289 (+ 2022us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:33:43.434308 (+ 19us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:33:43.435086 (+ 778us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:33:43.436057 (+ 971us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:33:43.436092 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:33:43.438797 (+ 2705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:33:43.438804 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:33:43.441015 (+ 2211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:33:43.441070 (+ 55us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:33:43.442714 (+ 1644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:33:43.442725 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:33:43.443086 (+ 361us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:33:43.444048 (+ 962us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:33:43.444084 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:33:43.446951 (+ 2867us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:33:43.451659 (+ 4708us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:33:43.451671 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:33:43.451694 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:33:43.452363 (+ 669us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:33:43.453042 (+ 679us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:33:43.453048 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:33:43.453052 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:33:43.453357 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:33:43.454170 (+ 813us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:33:43.454179 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:33:43.454659 (+ 480us) client_negotiation.cc:769] Sending connection context
0810 11:33:43.454994 (+ 335us) client_negotiation.cc:240] Negotiation successful
0810 11:33:43.455623 (+ 629us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2325,"thread_start_us":2091,"threads_started":1}
I20260810 11:33:43.459636 21879 heartbeater.cc:347] Connected to a master server at 127.8.18.254:36507
I20260810 11:33:43.460043 21879 heartbeater.cc:464] Registering TS with master...
I20260810 11:33:43.461050 21879 heartbeater.cc:511] Master 127.8.18.254:36507 requested a full tablet report, sending...
I20260810 11:33:43.463682 21534 ts_manager.cc:194] Registered new tserver with Master: 94767f4c56104676b69351a3d83c31f9 (127.8.18.194:44583)
I20260810 11:33:43.466199 21534 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.18.194@KRBTEST.COM'} at 127.8.18.194:53199
I20260810 11:33:43.749289 21734 heartbeater.cc:464] Registering TS with master...
I20260810 11:33:43.749902 21734 heartbeater.cc:511] Master 127.8.18.254:36507 requested a full tablet report, sending...
I20260810 11:33:43.751766 21534 ts_manager.cc:194] Registered new tserver with Master: fb558a810efd415cb0bf25c474b3bf99 (127.8.18.193:45081)
I20260810 11:33:43.753171 21534 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.18.193@KRBTEST.COM'} at 127.8.18.193:60153
Aug 10 11:33:43 dist-test-slave-srgv krb5kdc[20509](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361623, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.254@KRBTEST.COM for HTTP/127.8.18.212@KRBTEST.COM
W20260810 11:33:43.918907 21883 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:33:43.919368 21883 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:33:43.919553 21883 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:33:43.933010 21883 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:33:43.933215 21883 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:33:43.933461 21883 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.18.195
I20260810 11:33:43.950487 21883 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.18.212:33245
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.8.18.212:41093/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.195.keytab
--principal=kudu/127.8.18.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.18.195:44519
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.18.195
--webserver_port=34233
--tserver_master_addrs=127.8.18.254:36507
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.18.195
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:33:43.952400 21883 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:33:43.954298 21883 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:33:43.973496 21890 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:33:43.979856 21889 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:33:43.987866 21892 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:33:43.990027 21883 server_base.cc:1034] running on GCE node
I20260810 11:33:43.990988 21883 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:33:43.992369 21883 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:33:43.993948 21883 hybrid_clock.cc:648] HybridClock initialized: now 1786361623993761 us; error 71 us; skew 500 ppm
I20260810 11:33:43.994432 21883 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:33:43 dist-test-slave-srgv krb5kdc[20509](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361623, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:33:43.999058 21883 init.cc:377] Logged in from keytab as kudu/127.8.18.195@KRBTEST.COM (short username kudu)
I20260810 11:33:44.001466 21883 webserver.cc:460] Webserver started at http://127.8.18.195:34233/ using document root <none> and password file <none>
I20260810 11:33:44.002552 21883 fs_manager.cc:359] Metadata directory not provided
I20260810 11:33:44.002732 21883 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 10 11:33:44 dist-test-slave-srgv krb5kdc[20509](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361623, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.195@KRBTEST.COM for HTTP/127.8.18.212@KRBTEST.COM
I20260810 11:33:44.037864 21883 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.005s sys 0.000s
I20260810 11:33:44.042212 21900 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:33:44.044533 21883 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.001s
I20260810 11:33:44.044845 21883 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/minicluster-data/ts-2/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/minicluster-data/ts-2/wal
uuid: "584fbc0fd7f0407a9ea8811cf932d743"
format_stamp: "Formatted at 2026-08-10 11:33:40 on dist-test-slave-srgv"
server_key: "8b67f1fe9319dbdcb5ca43fb33f2280a"
server_key_iv: "42f2d30358c6663180eeed516a6db927"
server_key_version: "kuduclusterkey@0"
I20260810 11:33:44.046542 21883 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:33:44.065315 21883 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:33:44.071700 21883 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:33:44.072115 21883 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:33:44.073725 21883 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:33:44.075863 21883 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:33:44.076066 21883 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260810 11:33:44.076220 21883 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:33:44.076321 21883 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:33:44.154582 21883 rpc_server.cc:307] RPC server started. Bound to: 127.8.18.195:44519
I20260810 11:33:44.154831 22014 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.18.195:44519 every 8 connection(s)
I20260810 11:33:44.156787 21883 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/minicluster-data/ts-2/data/info.pb
I20260810 11:33:44.164829 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 21883
Aug 10 11:33:44 dist-test-slave-srgv krb5kdc[20509](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361623, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.195@KRBTEST.COM for kudu/127.8.18.254@KRBTEST.COM
I20260810 11:33:44.204653 22017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:33:44.168060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55346 (local address 127.8.18.254:36507)
0810 11:33:44.184418 (+ 16358us) server_negotiation.cc:206] Beginning negotiation
0810 11:33:44.184425 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:33:44.184447 (+ 22us) server_negotiation.cc:407] Connection header received
0810 11:33:44.184517 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:33:44.184524 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:33:44.184622 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:33:44.184786 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:33:44.187353 (+ 2567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:33:44.188473 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:33:44.190745 (+ 2272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:33:44.191465 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:33:44.191572 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:33:44.195726 (+ 4154us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:33:44.195733 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:33:44.195737 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:33:44.195798 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:33:44.199720 (+ 3922us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:33:44.200820 (+ 1100us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:33:44.200825 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:33:44.200828 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:33:44.201024 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:33:44.201968 (+ 944us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:33:44.201976 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:33:44.201980 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:33:44.202434 (+ 454us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:33:44.202727 (+ 293us) server_negotiation.cc:1035] Waiting for connection context
0810 11:33:44.203830 (+ 1103us) server_negotiation.cc:299] Negotiation successful
0810 11:33:44.204164 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16208,"spinlock_wait_cycles":6016,"thread_start_us":418,"threads_started":1}
I20260810 11:33:44.211004 22018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:33:44.167088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:36507 (local address 127.8.18.195:34489)
0810 11:33:44.183456 (+ 16368us) negotiation.cc:107] Waiting for socket to connect
0810 11:33:44.183548 (+ 92us) client_negotiation.cc:174] Beginning negotiation
0810 11:33:44.186368 (+ 2820us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:33:44.188024 (+ 1656us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:33:44.188039 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:33:44.189258 (+ 1219us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:33:44.190220 (+ 962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:33:44.190253 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:33:44.192017 (+ 1764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:33:44.192024 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:33:44.194507 (+ 2483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:33:44.194544 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:33:44.196116 (+ 1572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:33:44.196123 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:33:44.196459 (+ 336us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:33:44.197232 (+ 773us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:33:44.197265 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:33:44.200414 (+ 3149us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:33:44.205456 (+ 5042us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:33:44.205472 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:33:44.205497 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:33:44.206245 (+ 748us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:33:44.206767 (+ 522us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:33:44.206772 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:33:44.206775 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:33:44.207146 (+ 371us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:33:44.207893 (+ 747us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:33:44.207901 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:33:44.208373 (+ 472us) client_negotiation.cc:769] Sending connection context
0810 11:33:44.208749 (+ 376us) client_negotiation.cc:240] Negotiation successful
0810 11:33:44.209430 (+ 681us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15914,"thread_start_us":15624,"threads_started":1}
I20260810 11:33:44.211721 22019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:33:44.166340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.195:34489 (local address 127.8.18.254:36507)
0810 11:33:44.168646 (+ 2306us) server_negotiation.cc:206] Beginning negotiation
0810 11:33:44.168656 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 11:33:44.183734 (+ 15078us) server_negotiation.cc:407] Connection header received
0810 11:33:44.186941 (+ 3207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:33:44.186948 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:33:44.187056 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:33:44.187142 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:33:44.190523 (+ 3381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:33:44.191805 (+ 1282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:33:44.194790 (+ 2985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:33:44.195561 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:33:44.195676 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:33:44.200808 (+ 5132us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:33:44.200816 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:33:44.200822 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:33:44.200886 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:33:44.205187 (+ 4301us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:33:44.206403 (+ 1216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:33:44.206409 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:33:44.206413 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:33:44.206577 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:33:44.207308 (+ 731us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:33:44.207313 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:33:44.207316 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:33:44.207648 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:33:44.207760 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0810 11:33:44.208873 (+ 1113us) server_negotiation.cc:299] Negotiation successful
0810 11:33:44.211038 (+ 2165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2128,"spinlock_wait_cycles":4864,"thread_start_us":17736,"threads_started":1}
I20260810 11:33:44.213138 22015 heartbeater.cc:347] Connected to a master server at 127.8.18.254:36507
I20260810 11:33:44.213548 22015 heartbeater.cc:464] Registering TS with master...
I20260810 11:33:44.214530 22015 heartbeater.cc:511] Master 127.8.18.254:36507 requested a full tablet report, sending...
I20260810 11:33:44.216406 21534 ts_manager.cc:194] Registered new tserver with Master: 584fbc0fd7f0407a9ea8811cf932d743 (127.8.18.195:44519)
Aug 10 11:33:44 dist-test-slave-srgv krb5kdc[20509](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361623, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.254@KRBTEST.COM for HTTP/127.8.18.212@KRBTEST.COM
I20260810 11:33:44.217625 21534 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.18.195@KRBTEST.COM'} at 127.8.18.195:34489
I20260810 11:33:44.224263 8267 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 11:33:44.258584 22019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:33:44.235755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55348 (local address 127.8.18.254:36507)
0810 11:33:44.236025 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0810 11:33:44.236034 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 11:33:44.237690 (+ 1656us) server_negotiation.cc:407] Connection header received
0810 11:33:44.237894 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:33:44.237903 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:33:44.238023 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:33:44.238207 (+ 184us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:33:44.241527 (+ 3320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:33:44.242713 (+ 1186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:33:44.244856 (+ 2143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:33:44.245584 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:33:44.245719 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:33:44.250030 (+ 4311us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:33:44.250038 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:33:44.250043 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:33:44.250099 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:33:44.254483 (+ 4384us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:33:44.255708 (+ 1225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:33:44.255716 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:33:44.255719 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:33:44.255931 (+ 212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:33:44.256482 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:33:44.256488 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:33:44.256491 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:33:44.256933 (+ 442us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:33:44.257190 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0810 11:33:44.257935 (+ 745us) server_negotiation.cc:299] Negotiation successful
0810 11:33:44.258240 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260810 11:33:44.267138 21534 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55348:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 11:33:44.279345 21534 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 11:33:44.383172 22032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:33:44.347153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.193:45081 (local address 127.0.0.1:45288)
0810 11:33:44.358906 (+ 11753us) negotiation.cc:107] Waiting for socket to connect
0810 11:33:44.358955 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0810 11:33:44.359211 (+ 256us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:33:44.363733 (+ 4522us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:33:44.363740 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:33:44.363779 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:33:44.370469 (+ 6690us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:33:44.370496 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:33:44.373514 (+ 3018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:33:44.373524 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:33:44.376852 (+ 3328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:33:44.376887 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:33:44.381996 (+ 5109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:33:44.382004 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:33:44.382257 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:33:44.382332 (+ 75us) client_negotiation.cc:769] Sending connection context
0810 11:33:44.382480 (+ 148us) client_negotiation.cc:240] Negotiation successful
0810 11:33:44.382729 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11423,"mutex_wait_us":3,"thread_start_us":15736,"threads_started":1}
I20260810 11:33:44.385478 22031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:33:44.347656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.194:44583 (local address 127.0.0.1:59350)
0810 11:33:44.362566 (+ 14910us) negotiation.cc:107] Waiting for socket to connect
0810 11:33:44.362586 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 11:33:44.362787 (+ 201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:33:44.363801 (+ 1014us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:33:44.363805 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:33:44.363831 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:33:44.364669 (+ 838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:33:44.364687 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:33:44.367539 (+ 2852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:33:44.367548 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:33:44.381705 (+ 14157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:33:44.381723 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:33:44.384422 (+ 2699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:33:44.384430 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:33:44.384648 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:33:44.384685 (+ 37us) client_negotiation.cc:769] Sending connection context
0810 11:33:44.384774 (+ 89us) client_negotiation.cc:240] Negotiation successful
0810 11:33:44.384942 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14800,"mutex_wait_us":184,"thread_start_us":10761,"threads_started":1}
I20260810 11:33:44.387214 22034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:33:44.348675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59350 (local address 127.8.18.194:44583)
0810 11:33:44.361460 (+ 12785us) server_negotiation.cc:206] Beginning negotiation
0810 11:33:44.361469 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 11:33:44.362734 (+ 1265us) server_negotiation.cc:407] Connection header received
0810 11:33:44.362980 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:33:44.362992 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:33:44.363295 (+ 303us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:33:44.363511 (+ 216us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:33:44.364945 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:33:44.367203 (+ 2258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:33:44.381893 (+ 14690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:33:44.383571 (+ 1678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:33:44.386111 (+ 2540us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:33:44.386402 (+ 291us) server_negotiation.cc:1035] Waiting for connection context
0810 11:33:44.386492 (+ 90us) server_negotiation.cc:299] Negotiation successful
0810 11:33:44.386740 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12616,"thread_start_us":12525,"threads_started":1}
I20260810 11:33:44.390828 22033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:33:44.348334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45288 (local address 127.8.18.193:45081)
0810 11:33:44.359941 (+ 11607us) server_negotiation.cc:206] Beginning negotiation
0810 11:33:44.359955 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 11:33:44.359998 (+ 43us) server_negotiation.cc:407] Connection header received
0810 11:33:44.360135 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:33:44.360147 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:33:44.360505 (+ 358us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:33:44.360792 (+ 287us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:33:44.370860 (+ 10068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:33:44.373169 (+ 2309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:33:44.377252 (+ 4083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:33:44.379642 (+ 2390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:33:44.379885 (+ 243us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:33:44.380267 (+ 382us) server_negotiation.cc:1035] Waiting for connection context
0810 11:33:44.390176 (+ 9909us) server_negotiation.cc:299] Negotiation successful
0810 11:33:44.390409 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11395,"thread_start_us":11252,"threads_started":1}
I20260810 11:33:44.392036 21813 tablet_service.cc:1467] Processing CreateTablet for tablet 931bddf51779424b8877c7685de10ef4 (DEFAULT_TABLE table=test-table [id=5e2a0975118a44c5bfa9d8032f8401e9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:33:44.393867 21813 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 931bddf51779424b8877c7685de10ef4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:33:44.395062 21666 tablet_service.cc:1467] Processing CreateTablet for tablet 931bddf51779424b8877c7685de10ef4 (DEFAULT_TABLE table=test-table [id=5e2a0975118a44c5bfa9d8032f8401e9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:33:44.396479 21666 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 931bddf51779424b8877c7685de10ef4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:33:44.406629 22035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:33:44.363416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.195:44519 (local address 127.0.0.1:51890)
0810 11:33:44.380671 (+ 17255us) negotiation.cc:107] Waiting for socket to connect
0810 11:33:44.380694 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0810 11:33:44.381227 (+ 533us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:33:44.385015 (+ 3788us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:33:44.385021 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:33:44.385057 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:33:44.385402 (+ 345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:33:44.385444 (+ 42us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:33:44.387691 (+ 2247us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:33:44.387697 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:33:44.389788 (+ 2091us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:33:44.389808 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:33:44.405640 (+ 15832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:33:44.405650 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:33:44.405924 (+ 274us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:33:44.405973 (+ 49us) client_negotiation.cc:769] Sending connection context
0810 11:33:44.406080 (+ 107us) client_negotiation.cc:240] Negotiation successful
0810 11:33:44.406247 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17098,"thread_start_us":16907,"threads_started":1}
I20260810 11:33:44.408108 22036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:33:44.363770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51890 (local address 127.8.18.195:44519)
0810 11:33:44.383906 (+ 20136us) server_negotiation.cc:206] Beginning negotiation
0810 11:33:44.383916 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 11:33:44.383957 (+ 41us) server_negotiation.cc:407] Connection header received
0810 11:33:44.384100 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:33:44.384109 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:33:44.384433 (+ 324us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:33:44.384687 (+ 254us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:33:44.385684 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:33:44.387457 (+ 1773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:33:44.390029 (+ 2572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:33:44.391642 (+ 1613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:33:44.391831 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:33:44.407065 (+ 15234us) server_negotiation.cc:1035] Waiting for connection context
0810 11:33:44.407337 (+ 272us) server_negotiation.cc:299] Negotiation successful
0810 11:33:44.407684 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19930,"thread_start_us":23709,"threads_started":1}
I20260810 11:33:44.421945 21949 tablet_service.cc:1467] Processing CreateTablet for tablet 931bddf51779424b8877c7685de10ef4 (DEFAULT_TABLE table=test-table [id=5e2a0975118a44c5bfa9d8032f8401e9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:33:44.423904 21949 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 931bddf51779424b8877c7685de10ef4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:33:44.437060 22037 tablet_bootstrap.cc:492] T 931bddf51779424b8877c7685de10ef4 P 94767f4c56104676b69351a3d83c31f9: Bootstrap starting.
I20260810 11:33:44.445731 22037 tablet_bootstrap.cc:654] T 931bddf51779424b8877c7685de10ef4 P 94767f4c56104676b69351a3d83c31f9: Neither blocks nor log segments found. Creating new log.
I20260810 11:33:44.447381 22037 log.cc:824] T 931bddf51779424b8877c7685de10ef4 P 94767f4c56104676b69351a3d83c31f9: Log is configured to *not* fsync() on all Append() calls
I20260810 11:33:44.448992 22038 tablet_bootstrap.cc:492] T 931bddf51779424b8877c7685de10ef4 P fb558a810efd415cb0bf25c474b3bf99: Bootstrap starting.
I20260810 11:33:44.455891 22038 tablet_bootstrap.cc:654] T 931bddf51779424b8877c7685de10ef4 P fb558a810efd415cb0bf25c474b3bf99: Neither blocks nor log segments found. Creating new log.
I20260810 11:33:44.457370 22038 log.cc:824] T 931bddf51779424b8877c7685de10ef4 P fb558a810efd415cb0bf25c474b3bf99: Log is configured to *not* fsync() on all Append() calls
I20260810 11:33:44.461382 22037 tablet_bootstrap.cc:492] T 931bddf51779424b8877c7685de10ef4 P 94767f4c56104676b69351a3d83c31f9: No bootstrap required, opened a new log
I20260810 11:33:44.461750 22037 ts_tablet_manager.cc:1397] T 931bddf51779424b8877c7685de10ef4 P 94767f4c56104676b69351a3d83c31f9: Time spent bootstrapping tablet: real 0.028s user 0.005s sys 0.005s
I20260810 11:33:44.462898 22040 tablet_bootstrap.cc:492] T 931bddf51779424b8877c7685de10ef4 P 584fbc0fd7f0407a9ea8811cf932d743: Bootstrap starting.
I20260810 11:33:44.468457 22038 tablet_bootstrap.cc:492] T 931bddf51779424b8877c7685de10ef4 P fb558a810efd415cb0bf25c474b3bf99: No bootstrap required, opened a new log
I20260810 11:33:44.468818 22038 ts_tablet_manager.cc:1397] T 931bddf51779424b8877c7685de10ef4 P fb558a810efd415cb0bf25c474b3bf99: Time spent bootstrapping tablet: real 0.020s user 0.002s sys 0.005s
I20260810 11:33:44.471550 21879 heartbeater.cc:503] Master 127.8.18.254:36507 was elected leader, sending a full tablet report...
I20260810 11:33:44.473320 22037 raft_consensus.cc:348] T 931bddf51779424b8877c7685de10ef4 P 94767f4c56104676b69351a3d83c31f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94767f4c56104676b69351a3d83c31f9" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 44583 } } peers { permanent_uuid: "fb558a810efd415cb0bf25c474b3bf99" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 45081 } } peers { permanent_uuid: "584fbc0fd7f0407a9ea8811cf932d743" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 44519 } }
I20260810 11:33:44.473855 22037 raft_consensus.cc:374] T 931bddf51779424b8877c7685de10ef4 P 94767f4c56104676b69351a3d83c31f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:33:44.473958 22037 raft_consensus.cc:729] T 931bddf51779424b8877c7685de10ef4 P 94767f4c56104676b69351a3d83c31f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94767f4c56104676b69351a3d83c31f9, State: Initialized, Role: FOLLOWER
I20260810 11:33:44.474637 22037 consensus_queue.cc:260] T 931bddf51779424b8877c7685de10ef4 P 94767f4c56104676b69351a3d83c31f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94767f4c56104676b69351a3d83c31f9" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 44583 } } peers { permanent_uuid: "fb558a810efd415cb0bf25c474b3bf99" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 45081 } } peers { permanent_uuid: "584fbc0fd7f0407a9ea8811cf932d743" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 44519 } }
I20260810 11:33:44.479894 22038 raft_consensus.cc:348] T 931bddf51779424b8877c7685de10ef4 P fb558a810efd415cb0bf25c474b3bf99 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94767f4c56104676b69351a3d83c31f9" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 44583 } } peers { permanent_uuid: "fb558a810efd415cb0bf25c474b3bf99" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 45081 } } peers { permanent_uuid: "584fbc0fd7f0407a9ea8811cf932d743" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 44519 } }
I20260810 11:33:44.480470 22038 raft_consensus.cc:374] T 931bddf51779424b8877c7685de10ef4 P fb558a810efd415cb0bf25c474b3bf99 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:33:44.480597 22038 raft_consensus.cc:729] T 931bddf51779424b8877c7685de10ef4 P fb558a810efd415cb0bf25c474b3bf99 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb558a810efd415cb0bf25c474b3bf99, State: Initialized, Role: FOLLOWER
I20260810 11:33:44.484934 22040 tablet_bootstrap.cc:654] T 931bddf51779424b8877c7685de10ef4 P 584fbc0fd7f0407a9ea8811cf932d743: Neither blocks nor log segments found. Creating new log.
I20260810 11:33:44.486682 22040 log.cc:824] T 931bddf51779424b8877c7685de10ef4 P 584fbc0fd7f0407a9ea8811cf932d743: Log is configured to *not* fsync() on all Append() calls
I20260810 11:33:44.487200 22037 ts_tablet_manager.cc:1428] T 931bddf51779424b8877c7685de10ef4 P 94767f4c56104676b69351a3d83c31f9: Time spent starting tablet: real 0.025s user 0.016s sys 0.009s
I20260810 11:33:44.488842 22038 consensus_queue.cc:260] T 931bddf51779424b8877c7685de10ef4 P fb558a810efd415cb0bf25c474b3bf99 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94767f4c56104676b69351a3d83c31f9" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 44583 } } peers { permanent_uuid: "fb558a810efd415cb0bf25c474b3bf99" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 45081 } } peers { permanent_uuid: "584fbc0fd7f0407a9ea8811cf932d743" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 44519 } }
I20260810 11:33:44.492312 21734 heartbeater.cc:503] Master 127.8.18.254:36507 was elected leader, sending a full tablet report...
I20260810 11:33:44.493247 22038 ts_tablet_manager.cc:1428] T 931bddf51779424b8877c7685de10ef4 P fb558a810efd415cb0bf25c474b3bf99: Time spent starting tablet: real 0.024s user 0.012s sys 0.004s
I20260810 11:33:44.497861 22040 tablet_bootstrap.cc:492] T 931bddf51779424b8877c7685de10ef4 P 584fbc0fd7f0407a9ea8811cf932d743: No bootstrap required, opened a new log
I20260810 11:33:44.498207 22040 ts_tablet_manager.cc:1397] T 931bddf51779424b8877c7685de10ef4 P 584fbc0fd7f0407a9ea8811cf932d743: Time spent bootstrapping tablet: real 0.036s user 0.007s sys 0.005s
I20260810 11:33:44.509701 22040 raft_consensus.cc:348] T 931bddf51779424b8877c7685de10ef4 P 584fbc0fd7f0407a9ea8811cf932d743 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94767f4c56104676b69351a3d83c31f9" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 44583 } } peers { permanent_uuid: "fb558a810efd415cb0bf25c474b3bf99" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 45081 } } peers { permanent_uuid: "584fbc0fd7f0407a9ea8811cf932d743" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 44519 } }
I20260810 11:33:44.510253 22040 raft_consensus.cc:374] T 931bddf51779424b8877c7685de10ef4 P 584fbc0fd7f0407a9ea8811cf932d743 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:33:44.510358 22040 raft_consensus.cc:729] T 931bddf51779424b8877c7685de10ef4 P 584fbc0fd7f0407a9ea8811cf932d743 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 584fbc0fd7f0407a9ea8811cf932d743, State: Initialized, Role: FOLLOWER
I20260810 11:33:44.511113 22040 consensus_queue.cc:260] T 931bddf51779424b8877c7685de10ef4 P 584fbc0fd7f0407a9ea8811cf932d743 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94767f4c56104676b69351a3d83c31f9" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 44583 } } peers { permanent_uuid: "fb558a810efd415cb0bf25c474b3bf99" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 45081 } } peers { permanent_uuid: "584fbc0fd7f0407a9ea8811cf932d743" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 44519 } }
I20260810 11:33:44.514276 22015 heartbeater.cc:503] Master 127.8.18.254:36507 was elected leader, sending a full tablet report...
I20260810 11:33:44.515170 22040 ts_tablet_manager.cc:1428] T 931bddf51779424b8877c7685de10ef4 P 584fbc0fd7f0407a9ea8811cf932d743: Time spent starting tablet: real 0.017s user 0.013s sys 0.003s
W20260810 11:33:44.668421 21880 tablet.cc:2369] T 931bddf51779424b8877c7685de10ef4 P 94767f4c56104676b69351a3d83c31f9: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 11:33:44.668677 22016 tablet.cc:2369] T 931bddf51779424b8877c7685de10ef4 P 584fbc0fd7f0407a9ea8811cf932d743: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 11:33:44.685086 22045 raft_consensus.cc:482] T 931bddf51779424b8877c7685de10ef4 P 584fbc0fd7f0407a9ea8811cf932d743 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 11:33:44.685469 22045 raft_consensus.cc:504] T 931bddf51779424b8877c7685de10ef4 P 584fbc0fd7f0407a9ea8811cf932d743 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94767f4c56104676b69351a3d83c31f9" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 44583 } } peers { permanent_uuid: "fb558a810efd415cb0bf25c474b3bf99" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 45081 } } peers { permanent_uuid: "584fbc0fd7f0407a9ea8811cf932d743" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 44519 } }
I20260810 11:33:44.687525 22045 leader_election.cc:288] T 931bddf51779424b8877c7685de10ef4 P 584fbc0fd7f0407a9ea8811cf932d743 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 94767f4c56104676b69351a3d83c31f9 (127.8.18.194:44583), fb558a810efd415cb0bf25c474b3bf99 (127.8.18.193:45081)
W20260810 11:33:44.688362 21735 tablet.cc:2369] T 931bddf51779424b8877c7685de10ef4 P fb558a810efd415cb0bf25c474b3bf99: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 11:33:44.695101 22018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:33:44.688181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.194:44583 (local address 127.8.18.195:53663)
0810 11:33:44.688544 (+ 363us) negotiation.cc:107] Waiting for socket to connect
0810 11:33:44.688580 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0810 11:33:44.688722 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:33:44.689659 (+ 937us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:33:44.689664 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:33:44.689694 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:33:44.689973 (+ 279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:33:44.689985 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:33:44.691435 (+ 1450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:33:44.691440 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:33:44.692974 (+ 1534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:33:44.692989 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:33:44.694276 (+ 1287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:33:44.694282 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:33:44.694503 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:33:44.694537 (+ 34us) client_negotiation.cc:769] Sending connection context
0810 11:33:44.694651 (+ 114us) client_negotiation.cc:240] Negotiation successful
0810 11:33:44.694797 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"spinlock_wait_cycles":33536}
I20260810 11:33:44.695955 22034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:33:44.689047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.195:53663 (local address 127.8.18.194:44583)
0810 11:33:44.689273 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0810 11:33:44.689277 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 11:33:44.689296 (+ 19us) server_negotiation.cc:407] Connection header received
0810 11:33:44.689360 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:33:44.689364 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:33:44.689508 (+ 144us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:33:44.689623 (+ 115us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:33:44.690168 (+ 545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:33:44.691265 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:33:44.693200 (+ 1935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:33:44.694074 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:33:44.694275 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:33:44.694357 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0810 11:33:44.695509 (+ 1152us) server_negotiation.cc:299] Negotiation successful
0810 11:33:44.695671 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260810 11:33:44.696740 22033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:33:44.688280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.195:51419 (local address 127.8.18.193:45081)
0810 11:33:44.688532 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0810 11:33:44.688538 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:33:44.689703 (+ 1165us) server_negotiation.cc:407] Connection header received
0810 11:33:44.689918 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:33:44.689926 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:33:44.690026 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:33:44.690119 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:33:44.690735 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:33:44.691672 (+ 937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:33:44.694342 (+ 2670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:33:44.695700 (+ 1358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:33:44.695873 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:33:44.695948 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0810 11:33:44.696417 (+ 469us) server_negotiation.cc:299] Negotiation successful
0810 11:33:44.696570 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260810 11:33:44.696820 22053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:33:44.688709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.193:45081 (local address 127.8.18.195:51419)
0810 11:33:44.689577 (+ 868us) negotiation.cc:107] Waiting for socket to connect
0810 11:33:44.689592 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 11:33:44.689744 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:33:44.690177 (+ 433us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:33:44.690181 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:33:44.690205 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:33:44.690536 (+ 331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:33:44.690550 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:33:44.691840 (+ 1290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:33:44.691846 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:33:44.694122 (+ 2276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:33:44.694139 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:33:44.696009 (+ 1870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:33:44.696018 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:33:44.696213 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:33:44.696251 (+ 38us) client_negotiation.cc:769] Sending connection context
0810 11:33:44.696341 (+ 90us) client_negotiation.cc:240] Negotiation successful
0810 11:33:44.696444 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":759,"thread_start_us":558,"threads_started":1}
I20260810 11:33:44.697142 21833 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "931bddf51779424b8877c7685de10ef4" candidate_uuid: "584fbc0fd7f0407a9ea8811cf932d743" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94767f4c56104676b69351a3d83c31f9" is_pre_election: true
I20260810 11:33:44.697845 21833 raft_consensus.cc:2454] T 931bddf51779424b8877c7685de10ef4 P 94767f4c56104676b69351a3d83c31f9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 584fbc0fd7f0407a9ea8811cf932d743 in term 0.
I20260810 11:33:44.697978 21687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "931bddf51779424b8877c7685de10ef4" candidate_uuid: "584fbc0fd7f0407a9ea8811cf932d743" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb558a810efd415cb0bf25c474b3bf99" is_pre_election: true
I20260810 11:33:44.698514 21687 raft_consensus.cc:2454] T 931bddf51779424b8877c7685de10ef4 P fb558a810efd415cb0bf25c474b3bf99 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 584fbc0fd7f0407a9ea8811cf932d743 in term 0.
I20260810 11:33:44.698761 21902 leader_election.cc:302] T 931bddf51779424b8877c7685de10ef4 P 584fbc0fd7f0407a9ea8811cf932d743 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 584fbc0fd7f0407a9ea8811cf932d743, 94767f4c56104676b69351a3d83c31f9; no voters:
I20260810 11:33:44.699335 22045 raft_consensus.cc:2781] T 931bddf51779424b8877c7685de10ef4 P 584fbc0fd7f0407a9ea8811cf932d743 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 11:33:44.699519 22045 raft_consensus.cc:482] T 931bddf51779424b8877c7685de10ef4 P 584fbc0fd7f0407a9ea8811cf932d743 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 11:33:44.699630 22045 raft_consensus.cc:3037] T 931bddf51779424b8877c7685de10ef4 P 584fbc0fd7f0407a9ea8811cf932d743 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:33:44.701992 22045 raft_consensus.cc:504] T 931bddf51779424b8877c7685de10ef4 P 584fbc0fd7f0407a9ea8811cf932d743 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94767f4c56104676b69351a3d83c31f9" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 44583 } } peers { permanent_uuid: "fb558a810efd415cb0bf25c474b3bf99" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 45081 } } peers { permanent_uuid: "584fbc0fd7f0407a9ea8811cf932d743" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 44519 } }
I20260810 11:33:44.702687 22045 leader_election.cc:288] T 931bddf51779424b8877c7685de10ef4 P 584fbc0fd7f0407a9ea8811cf932d743 [CANDIDATE]: Term 1 election: Requested vote from peers 94767f4c56104676b69351a3d83c31f9 (127.8.18.194:44583), fb558a810efd415cb0bf25c474b3bf99 (127.8.18.193:45081)
I20260810 11:33:44.703333 21833 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "931bddf51779424b8877c7685de10ef4" candidate_uuid: "584fbc0fd7f0407a9ea8811cf932d743" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94767f4c56104676b69351a3d83c31f9"
I20260810 11:33:44.703382 21687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "931bddf51779424b8877c7685de10ef4" candidate_uuid: "584fbc0fd7f0407a9ea8811cf932d743" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb558a810efd415cb0bf25c474b3bf99"
I20260810 11:33:44.703598 21833 raft_consensus.cc:3037] T 931bddf51779424b8877c7685de10ef4 P 94767f4c56104676b69351a3d83c31f9 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:33:44.703603 21687 raft_consensus.cc:3037] T 931bddf51779424b8877c7685de10ef4 P fb558a810efd415cb0bf25c474b3bf99 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:33:44.706205 21687 raft_consensus.cc:2454] T 931bddf51779424b8877c7685de10ef4 P fb558a810efd415cb0bf25c474b3bf99 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 584fbc0fd7f0407a9ea8811cf932d743 in term 1.
I20260810 11:33:44.706323 21833 raft_consensus.cc:2454] T 931bddf51779424b8877c7685de10ef4 P 94767f4c56104676b69351a3d83c31f9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 584fbc0fd7f0407a9ea8811cf932d743 in term 1.
I20260810 11:33:44.706851 21904 leader_election.cc:302] T 931bddf51779424b8877c7685de10ef4 P 584fbc0fd7f0407a9ea8811cf932d743 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 584fbc0fd7f0407a9ea8811cf932d743, fb558a810efd415cb0bf25c474b3bf99; no voters:
I20260810 11:33:44.707247 22045 raft_consensus.cc:2781] T 931bddf51779424b8877c7685de10ef4 P 584fbc0fd7f0407a9ea8811cf932d743 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:33:44.707903 22045 raft_consensus.cc:686] T 931bddf51779424b8877c7685de10ef4 P 584fbc0fd7f0407a9ea8811cf932d743 [term 1 LEADER]: Becoming Leader. State: Replica: 584fbc0fd7f0407a9ea8811cf932d743, State: Running, Role: LEADER
I20260810 11:33:44.708462 22045 consensus_queue.cc:237] T 931bddf51779424b8877c7685de10ef4 P 584fbc0fd7f0407a9ea8811cf932d743 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94767f4c56104676b69351a3d83c31f9" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 44583 } } peers { permanent_uuid: "fb558a810efd415cb0bf25c474b3bf99" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 45081 } } peers { permanent_uuid: "584fbc0fd7f0407a9ea8811cf932d743" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 44519 } }
I20260810 11:33:44.713997 21533 catalog_manager.cc:5495] T 931bddf51779424b8877c7685de10ef4 P 584fbc0fd7f0407a9ea8811cf932d743 reported cstate change: term changed from 0 to 1, leader changed from <none> to 584fbc0fd7f0407a9ea8811cf932d743 (127.8.18.195). New cstate: current_term: 1 leader_uuid: "584fbc0fd7f0407a9ea8811cf932d743" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94767f4c56104676b69351a3d83c31f9" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 44583 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fb558a810efd415cb0bf25c474b3bf99" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 45081 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "584fbc0fd7f0407a9ea8811cf932d743" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 44519 } health_report { overall_health: HEALTHY } } }
I20260810 11:33:44.781183 22036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:33:44.774813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51898 (local address 127.8.18.195:44519)
0810 11:33:44.775006 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0810 11:33:44.775011 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:33:44.775308 (+ 297us) server_negotiation.cc:407] Connection header received
0810 11:33:44.775494 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:33:44.775498 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:33:44.775581 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:33:44.775729 (+ 148us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 11:33:44.776139 (+ 410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:33:44.776973 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:33:44.778648 (+ 1675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:33:44.779071 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:33:44.779205 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:33:44.779931 (+ 726us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 11:33:44.780511 (+ 580us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 11:33:44.780632 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0810 11:33:44.780814 (+ 182us) server_negotiation.cc:299] Negotiation successful
0810 11:33:44.780927 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260810 11:33:44.791512 21687 raft_consensus.cc:1261] T 931bddf51779424b8877c7685de10ef4 P fb558a810efd415cb0bf25c474b3bf99 [term 1 FOLLOWER]: Refusing update from remote peer 584fbc0fd7f0407a9ea8811cf932d743: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 11:33:44.791512 21833 raft_consensus.cc:1261] T 931bddf51779424b8877c7685de10ef4 P 94767f4c56104676b69351a3d83c31f9 [term 1 FOLLOWER]: Refusing update from remote peer 584fbc0fd7f0407a9ea8811cf932d743: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 11:33:44.792893 22054 consensus_queue.cc:1035] T 931bddf51779424b8877c7685de10ef4 P 584fbc0fd7f0407a9ea8811cf932d743 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94767f4c56104676b69351a3d83c31f9" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 44583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 11:33:44.793370 22045 consensus_queue.cc:1035] T 931bddf51779424b8877c7685de10ef4 P 584fbc0fd7f0407a9ea8811cf932d743 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb558a810efd415cb0bf25c474b3bf99" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 45081 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 11:33:44.807492 22058 mvcc.cc:205] Tried to move back new op lower bound from 7316937215132790784 to 7316937214810759168. Current Snapshot: MvccSnapshot[applied={T|T < 7316937215132790784}]
I20260810 11:33:44.808120 22060 mvcc.cc:205] Tried to move back new op lower bound from 7316937215132790784 to 7316937214810759168. Current Snapshot: MvccSnapshot[applied={T|T < 7316937215132790784}]
I20260810 11:33:44.827831 21533 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55348:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 11:33:44.828241 21533 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55348:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 11:33:44.833019 21533 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d482e868edcf44a3a887277ae45225c8: Sending DeleteTablet for 3 replicas of tablet 931bddf51779424b8877c7685de10ef4
I20260810 11:33:44.834560 21813 tablet_service.cc:1514] Processing DeleteTablet for tablet 931bddf51779424b8877c7685de10ef4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:33:44 UTC) from {username='kudu', principal='kudu/127.8.18.254@KRBTEST.COM'} at 127.0.0.1:59350
I20260810 11:33:44.834693 21949 tablet_service.cc:1514] Processing DeleteTablet for tablet 931bddf51779424b8877c7685de10ef4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:33:44 UTC) from {username='kudu', principal='kudu/127.8.18.254@KRBTEST.COM'} at 127.0.0.1:51890
I20260810 11:33:44.834977 21666 tablet_service.cc:1514] Processing DeleteTablet for tablet 931bddf51779424b8877c7685de10ef4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:33:44 UTC) from {username='kudu', principal='kudu/127.8.18.254@KRBTEST.COM'} at 127.0.0.1:45288
I20260810 11:33:44.837847 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 21572
I20260810 11:33:44.850682 22068 tablet_replica.cc:330] T 931bddf51779424b8877c7685de10ef4 P 94767f4c56104676b69351a3d83c31f9: stopping tablet replica
I20260810 11:33:44.850968 22070 tablet_replica.cc:330] T 931bddf51779424b8877c7685de10ef4 P fb558a810efd415cb0bf25c474b3bf99: stopping tablet replica
I20260810 11:33:44.851298 22068 raft_consensus.cc:2229] T 931bddf51779424b8877c7685de10ef4 P 94767f4c56104676b69351a3d83c31f9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 11:33:44.851548 22070 raft_consensus.cc:2229] T 931bddf51779424b8877c7685de10ef4 P fb558a810efd415cb0bf25c474b3bf99 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 11:33:44.851907 22068 raft_consensus.cc:2258] T 931bddf51779424b8877c7685de10ef4 P 94767f4c56104676b69351a3d83c31f9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:33:44.852268 22070 raft_consensus.cc:2258] T 931bddf51779424b8877c7685de10ef4 P fb558a810efd415cb0bf25c474b3bf99 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:33:44.855154 22070 ts_tablet_manager.cc:1905] T 931bddf51779424b8877c7685de10ef4 P fb558a810efd415cb0bf25c474b3bf99: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:33:44.861966 22069 tablet_replica.cc:330] T 931bddf51779424b8877c7685de10ef4 P 584fbc0fd7f0407a9ea8811cf932d743: stopping tablet replica
I20260810 11:33:44.862680 22069 raft_consensus.cc:2229] T 931bddf51779424b8877c7685de10ef4 P 584fbc0fd7f0407a9ea8811cf932d743 [term 1 LEADER]: Raft consensus shutting down.
I20260810 11:33:44.863478 22070 ts_tablet_manager.cc:1918] T 931bddf51779424b8877c7685de10ef4 P fb558a810efd415cb0bf25c474b3bf99: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 11:33:44.863512 22069 raft_consensus.cc:2258] T 931bddf51779424b8877c7685de10ef4 P 584fbc0fd7f0407a9ea8811cf932d743 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:33:44.863711 22070 log.cc:1196] T 931bddf51779424b8877c7685de10ef4 P fb558a810efd415cb0bf25c474b3bf99: Deleting WAL directory at /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/minicluster-data/ts-0/wal/wals/931bddf51779424b8877c7685de10ef4
I20260810 11:33:44.866144 22069 ts_tablet_manager.cc:1905] T 931bddf51779424b8877c7685de10ef4 P 584fbc0fd7f0407a9ea8811cf932d743: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:33:44.866139 22068 ts_tablet_manager.cc:1905] T 931bddf51779424b8877c7685de10ef4 P 94767f4c56104676b69351a3d83c31f9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:33:44.873039 22070 ts_tablet_manager.cc:1939] T 931bddf51779424b8877c7685de10ef4 P fb558a810efd415cb0bf25c474b3bf99: Deleting consensus metadata
I20260810 11:33:44.873987 22033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:33:44.839360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45292 (local address 127.8.18.193:45081)
0810 11:33:44.839631 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0810 11:33:44.839641 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 11:33:44.849923 (+ 10282us) server_negotiation.cc:407] Connection header received
0810 11:33:44.850212 (+ 289us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:33:44.850223 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:33:44.850410 (+ 187us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:33:44.850614 (+ 204us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:33:44.853775 (+ 3161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:33:44.855341 (+ 1566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:33:44.858256 (+ 2915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:33:44.859202 (+ 946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:33:44.859344 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:33:44.863749 (+ 4405us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:33:44.863768 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:33:44.863786 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:33:44.864294 (+ 508us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:33:44.868883 (+ 4589us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:33:44.870094 (+ 1211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:33:44.870106 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:33:44.870110 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:33:44.870308 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:33:44.870859 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:33:44.870863 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:33:44.870867 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:33:44.871422 (+ 555us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:33:44.872920 (+ 1498us) server_negotiation.cc:1035] Waiting for connection context
0810 11:33:44.873119 (+ 199us) server_negotiation.cc:299] Negotiation successful
0810 11:33:44.873537 (+ 418us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260810 11:33:44.874100 22069 ts_tablet_manager.cc:1918] T 931bddf51779424b8877c7685de10ef4 P 584fbc0fd7f0407a9ea8811cf932d743: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 11:33:44.874315 22069 log.cc:1196] T 931bddf51779424b8877c7685de10ef4 P 584fbc0fd7f0407a9ea8811cf932d743: Deleting WAL directory at /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/minicluster-data/ts-2/wal/wals/931bddf51779424b8877c7685de10ef4
I20260810 11:33:44.874894 22069 ts_tablet_manager.cc:1939] T 931bddf51779424b8877c7685de10ef4 P 584fbc0fd7f0407a9ea8811cf932d743: Deleting consensus metadata
I20260810 11:33:44.875159 21728 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:33:44.875381 21521 catalog_manager.cc:4841] TS fb558a810efd415cb0bf25c474b3bf99 (127.8.18.193:45081): tablet 931bddf51779424b8877c7685de10ef4 (table test-table [id=5e2a0975118a44c5bfa9d8032f8401e9]) successfully deleted
I20260810 11:33:44.876734 21521 catalog_manager.cc:4841] TS 584fbc0fd7f0407a9ea8811cf932d743 (127.8.18.195:44519): tablet 931bddf51779424b8877c7685de10ef4 (table test-table [id=5e2a0975118a44c5bfa9d8032f8401e9]) successfully deleted
I20260810 11:33:44.877645 22068 ts_tablet_manager.cc:1918] T 931bddf51779424b8877c7685de10ef4 P 94767f4c56104676b69351a3d83c31f9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 11:33:44.877902 22068 log.cc:1196] T 931bddf51779424b8877c7685de10ef4 P 94767f4c56104676b69351a3d83c31f9: Deleting WAL directory at /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786361401922362-8267-0/minicluster-data/ts-1/wal/wals/931bddf51779424b8877c7685de10ef4
I20260810 11:33:44.879118 22068 ts_tablet_manager.cc:1939] T 931bddf51779424b8877c7685de10ef4 P 94767f4c56104676b69351a3d83c31f9: Deleting consensus metadata
I20260810 11:33:44.881733 21519 catalog_manager.cc:4841] TS 94767f4c56104676b69351a3d83c31f9 (127.8.18.194:44583): tablet 931bddf51779424b8877c7685de10ef4 (table test-table [id=5e2a0975118a44c5bfa9d8032f8401e9]) successfully deleted
I20260810 11:33:44.972826 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 21572
I20260810 11:33:44.993630 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 21738
I20260810 11:33:45.011819 22034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:33:44.994546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59366 (local address 127.8.18.194:44583)
0810 11:33:44.994743 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0810 11:33:44.994747 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 11:33:44.994772 (+ 25us) server_negotiation.cc:407] Connection header received
0810 11:33:44.995044 (+ 272us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:33:44.995049 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:33:44.995122 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:33:44.995239 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:33:44.997221 (+ 1982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:33:44.998041 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:33:45.000177 (+ 2136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:33:45.000618 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:33:45.000746 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:33:45.004519 (+ 3773us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:33:45.004537 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:33:45.004554 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:33:45.004639 (+ 85us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:33:45.008444 (+ 3805us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:33:45.009652 (+ 1208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:33:45.009661 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:33:45.009663 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:33:45.009844 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:33:45.010327 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:33:45.010330 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:33:45.010332 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:33:45.010730 (+ 398us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:33:45.010968 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0810 11:33:45.011352 (+ 384us) server_negotiation.cc:299] Negotiation successful
0810 11:33:45.011542 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
I20260810 11:33:45.012629 21874 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:33:45.130950 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 21738
I20260810 11:33:45.149567 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 21883
I20260810 11:33:45.171101 22036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:33:45.150444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51904 (local address 127.8.18.195:44519)
0810 11:33:45.150753 (+ 309us) server_negotiation.cc:206] Beginning negotiation
0810 11:33:45.150760 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:33:45.150787 (+ 27us) server_negotiation.cc:407] Connection header received
0810 11:33:45.150882 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:33:45.150888 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:33:45.150979 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:33:45.151109 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:33:45.152634 (+ 1525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:33:45.154235 (+ 1601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:33:45.156378 (+ 2143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:33:45.157182 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:33:45.157365 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:33:45.161166 (+ 3801us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:33:45.161183 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:33:45.161203 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:33:45.161275 (+ 72us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:33:45.167018 (+ 5743us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:33:45.168230 (+ 1212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:33:45.168251 (+ 21us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:33:45.168256 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:33:45.168478 (+ 222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:33:45.168985 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:33:45.168991 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:33:45.168995 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:33:45.169581 (+ 586us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:33:45.170092 (+ 511us) server_negotiation.cc:1035] Waiting for connection context
0810 11:33:45.170305 (+ 213us) server_negotiation.cc:299] Negotiation successful
0810 11:33:45.170599 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137}
I20260810 11:33:45.172009 22009 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:33:45.299099 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 21883
I20260810 11:33:45.321105 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 21501
I20260810 11:33:45.322531 21565 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:33:45.441766 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 21501
I20260810 11:33:45.469866 8267 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260810 11:33:45.817565 8267 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260810 11:33:45.817878 8267 mini_ranger.cc:66] Stopping Ranger...
I20260810 11:33:46.165904 8267 mini_ranger.cc:68] Stopped Ranger
2026-08-10 11:33:46.166 UTC [20539] LOG: received smart shutdown request
2026-08-10 11:33:46.170 UTC [20539] LOG: background worker "logical replication launcher" (PID 20548) exited with exit code 1
2026-08-10 11:33:46.170 UTC [20543] LOG: shutting down
2026-08-10 11:33:46.367 UTC [20539] LOG: database system is shut down
2026-08-10T11:33:46Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (61056 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786361401922362-8267-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 11:33:46 dist-test-slave-srgv krb5kdc[22084](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 11:33:46 dist-test-slave-srgv krb5kdc[22084](info): set up 2 sockets
Aug 10 11:33:46 dist-test-slave-srgv krb5kdc[22084](info): commencing operation
krb5kdc: starting...
W20260810 11:33:47.680045 8267 mini_kdc.cc:121] Time spent starting KDC: real 1.131s user 0.000s sys 0.039s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786361401922362-8267-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786361401922362-8267-0/krb5kdc/test-user.keytab.
Aug 10 11:33:47 dist-test-slave-srgv krb5kdc[22084](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361627, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T11:33:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T11:33:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.18.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.18.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.18.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.254.keytab.
Entry for principal kudu/127.8.18.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.254.keytab.
I20260810 11:33:47.998589 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786361401922362-8267-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786361401922362-8267-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786361401922362-8267-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.18.254:39411
--webserver_interface=127.8.18.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.254.keytab
--principal=kudu/127.8.18.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.18.212:33883
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.18.254:39411,127.8.18.253:39493,127.8.18.252:39373
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786361401922362-8267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786361401922362-8267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786361401922362-8267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:33:48.389827 22097 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:33:48.390254 22097 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:33:48.390429 22097 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:33:48.398626 22097 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 11:33:48.398804 22097 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:33:48.398924 22097 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:33:48.399026 22097 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 11:33:48.399137 22097 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 11:33:48.410229 22097 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.18.212:33883
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786361401922362-8267-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786361401922362-8267-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.18.254:39411,127.8.18.253:39493,127.8.18.252:39373
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.254.keytab
--principal=kudu/127.8.18.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.18.254:39411
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.18.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786361401922362-8267-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:33:48.411792 22097 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:33:48.413239 22097 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:33:48.427465 22105 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:33:48.427579 22103 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:33:48.427560 22102 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:33:48.431268 22097 server_base.cc:1034] running on GCE node
I20260810 11:33:48.432288 22097 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:33:48.434113 22097 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:33:48.435392 22097 hybrid_clock.cc:648] HybridClock initialized: now 1786361628435357 us; error 69 us; skew 500 ppm
I20260810 11:33:48.435817 22097 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:33:48 dist-test-slave-srgv krb5kdc[22084](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361628, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:33:48.439569 22097 init.cc:377] Logged in from keytab as kudu/127.8.18.254@KRBTEST.COM (short username kudu)
I20260810 11:33:48.441756 22097 webserver.cc:460] Webserver started at http://127.8.18.254:35345/ using document root <none> and password file <none>
I20260810 11:33:48.442605 22097 fs_manager.cc:359] Metadata directory not provided
I20260810 11:33:48.442778 22097 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:33:48.443127 22097 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:33:48.446919 22097 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786361401922362-8267-0/minicluster-data/master-0/data/instance:
uuid: "837a7216db5341ceb09244a6dd0b6c98"
format_stamp: "Formatted at 2026-08-10 11:33:48 on dist-test-slave-srgv"
server_key: "b601b1bd7874907826f75cebd31e2eeb"
server_key_iv: "5b5497dd409427390e9a47bc36105c33"
server_key_version: "encryptionkey@0"
I20260810 11:33:48.447989 22097 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786361401922362-8267-0/minicluster-data/master-0/wal/instance:
uuid: "837a7216db5341ceb09244a6dd0b6c98"
format_stamp: "Formatted at 2026-08-10 11:33:48 on dist-test-slave-srgv"
server_key: "b601b1bd7874907826f75cebd31e2eeb"
server_key_iv: "5b5497dd409427390e9a47bc36105c33"
server_key_version: "encryptionkey@0"
I20260810 11:33:48.454869 22097 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260810 11:33:48.459352 22112 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:33:48.461095 22097 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 11:33:48.461344 22097 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786361401922362-8267-0/minicluster-data/master-0/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786361401922362-8267-0/minicluster-data/master-0/wal
uuid: "837a7216db5341ceb09244a6dd0b6c98"
format_stamp: "Formatted at 2026-08-10 11:33:48 on dist-test-slave-srgv"
server_key: "b601b1bd7874907826f75cebd31e2eeb"
server_key_iv: "5b5497dd409427390e9a47bc36105c33"
server_key_version: "encryptionkey@0"
I20260810 11:33:48.461681 22097 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786361401922362-8267-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786361401922362-8267-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786361401922362-8267-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:33:48.486436 22097 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:33:48.492105 22097 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:33:48.492413 22097 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:33:48.515475 22097 rpc_server.cc:307] RPC server started. Bound to: 127.8.18.254:39411
I20260810 11:33:48.515527 22165 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.18.254:39411 every 8 connection(s)
I20260810 11:33:48.517103 22097 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
I20260810 11:33:48.521384 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 22097
I20260810 11:33:48.521688 22166 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:33:48.521747 8267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786361401922362-8267-0/minicluster-data/master-0/wal/instance
I20260810 11:33:48.522436 8267 external_mini_cluster.cc:1428] Setting key 9c2b9b97525eba520cdd76c1f93404c1
I20260810 11:33:48.530426 22166 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.18.254" port: 39411 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.8.18.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.18.253@KRBTEST.COM" created.
Aug 10 11:33:48 dist-test-slave-srgv krb5kdc[22084](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361628, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.254@KRBTEST.COM for kudu/127.8.18.254@KRBTEST.COM
I20260810 11:33:48.562253 22169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:33:48.534385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:39411 (local address 127.0.0.1:58344)
0810 11:33:48.535501 (+ 1116us) negotiation.cc:107] Waiting for socket to connect
0810 11:33:48.535557 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0810 11:33:48.538119 (+ 2562us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:33:48.539192 (+ 1073us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:33:48.539200 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:33:48.540169 (+ 969us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:33:48.540583 (+ 414us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:33:48.540606 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:33:48.542851 (+ 2245us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:33:48.542856 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:33:48.544180 (+ 1324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:33:48.544198 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:33:48.545549 (+ 1351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:33:48.545558 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:33:48.545784 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:33:48.546625 (+ 841us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:33:48.546663 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:33:48.552797 (+ 6134us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:33:48.556971 (+ 4174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:33:48.556988 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:33:48.557002 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:33:48.557688 (+ 686us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:33:48.558211 (+ 523us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:33:48.558216 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:33:48.558222 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:33:48.558403 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:33:48.559292 (+ 889us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:33:48.559309 (+ 17us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:33:48.559567 (+ 258us) client_negotiation.cc:769] Sending connection context
0810 11:33:48.559855 (+ 288us) client_negotiation.cc:240] Negotiation successful
0810 11:33:48.560566 (+ 711us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":799,"mutex_wait_us":176,"thread_start_us":508,"threads_started":1}
I20260810 11:33:48.562282 22170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:33:48.534386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58344 (local address 127.8.18.254:39411)
0810 11:33:48.536526 (+ 2140us) server_negotiation.cc:206] Beginning negotiation
0810 11:33:48.536536 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 11:33:48.536593 (+ 57us) server_negotiation.cc:407] Connection header received
0810 11:33:48.538619 (+ 2026us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:33:48.538626 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:33:48.538913 (+ 287us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:33:48.539168 (+ 255us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:33:48.540859 (+ 1691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:33:48.542611 (+ 1752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:33:48.544407 (+ 1796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:33:48.545147 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:33:48.545429 (+ 282us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:33:48.553133 (+ 7704us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:33:48.553145 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:33:48.553161 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:33:48.553215 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:33:48.556767 (+ 3552us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:33:48.557879 (+ 1112us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:33:48.557887 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:33:48.557890 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:33:48.558065 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:33:48.558536 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:33:48.558546 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:33:48.558549 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:33:48.559130 (+ 581us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:33:48.559402 (+ 272us) server_negotiation.cc:1035] Waiting for connection context
0810 11:33:48.560015 (+ 613us) server_negotiation.cc:299] Negotiation successful
0810 11:33:48.560420 (+ 405us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1870,"spinlock_wait_cycles":18688,"thread_start_us":1569,"threads_started":1}
I20260810 11:33:48.565975 22166 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.18.253" port: 39493 } has no permanent_uuid. Determining permanent_uuid...
I20260810 11:33:48.567315 22169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:33:48.566783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.253:39493 (local address 127.0.0.1:60368)
0810 11:33:48.567022 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0810 11:33:48.567167 (+ 145us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.253:39493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
W20260810 11:33:48.567754 22116 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.18.253:39493: connect: Connection refused (error 111)
I20260810 11:33:48.569792 22169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:33:48.569258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.253:39493 (local address 127.0.0.1:60370)
0810 11:33:48.569487 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0810 11:33:48.569576 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.253:39493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260810 11:33:48.570114 22166 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.18.253:39493: Network error: Client connection negotiation failed: client connection to 127.8.18.253:39493: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.18.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.253.keytab.
Entry for principal kudu/127.8.18.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.253.keytab.
I20260810 11:33:48.590281 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786361401922362-8267-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786361401922362-8267-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786361401922362-8267-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786361401922362-8267-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.18.253:39493
--webserver_interface=127.8.18.253
--webserver_port=0
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.253.keytab
--principal=kudu/127.8.18.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.18.212:33883
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.18.254:39411,127.8.18.253:39493,127.8.18.252:39373
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786361401922362-8267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786361401922362-8267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786361401922362-8267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 11:33:48.619656 22166 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.18.253" port: 39493 } attempt: 1
I20260810 11:33:48.621176 22169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:33:48.620653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.253:39493 (local address 127.0.0.1:60384)
0810 11:33:48.620903 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0810 11:33:48.621040 (+ 137us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.253:39493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260810 11:33:48.622356 22169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:33:48.621937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.253:39493 (local address 127.0.0.1:60386)
0810 11:33:48.622093 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0810 11:33:48.622174 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.253:39493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260810 11:33:48.622583 22166 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.18.253:39493: Network error: Client connection negotiation failed: client connection to 127.8.18.253:39493: connect: Connection refused (error 111)
I20260810 11:33:48.690923 22166 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.18.253" port: 39493 } attempt: 2
I20260810 11:33:48.692266 22169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:33:48.691732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.253:39493 (local address 127.0.0.1:60392)
0810 11:33:48.691979 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0810 11:33:48.692089 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.253:39493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102}
I20260810 11:33:48.693766 22169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:33:48.693205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.253:39493 (local address 127.0.0.1:60398)
0810 11:33:48.693390 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0810 11:33:48.693558 (+ 168us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.253:39493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
W20260810 11:33:48.694038 22166 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.18.253:39493: Network error: Client connection negotiation failed: client connection to 127.8.18.253:39493: connect: Connection refused (error 111)
I20260810 11:33:48.785384 22166 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.18.253" port: 39493 } attempt: 3
I20260810 11:33:48.786723 22169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:33:48.786173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.253:39493 (local address 127.0.0.1:60410)
0810 11:33:48.786445 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0810 11:33:48.786571 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.253:39493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125}
I20260810 11:33:48.788128 22169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:33:48.787678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.253:39493 (local address 127.0.0.1:60422)
0810 11:33:48.787850 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0810 11:33:48.787940 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.253:39493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260810 11:33:48.788403 22166 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.18.253:39493: Network error: Client connection negotiation failed: client connection to 127.8.18.253:39493: connect: Connection refused (error 111)
I20260810 11:33:48.931731 22166 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.18.253" port: 39493 } attempt: 4
I20260810 11:33:48.933286 22169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:33:48.932664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.253:39493 (local address 127.0.0.1:60430)
0810 11:33:48.933045 (+ 381us) negotiation.cc:107] Waiting for socket to connect
0810 11:33:48.933141 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.253:39493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":229}
I20260810 11:33:48.934501 22169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:33:48.934084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.253:39493 (local address 127.0.0.1:60438)
0810 11:33:48.934257 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0810 11:33:48.934364 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.253:39493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260810 11:33:48.934727 22166 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.18.253:39493: Network error: Client connection negotiation failed: client connection to 127.8.18.253:39493: connect: Connection refused (error 111)
W20260810 11:33:48.994573 22173 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:33:48.995021 22173 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:33:48.995216 22173 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:33:49.004258 22173 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 11:33:49.004465 22173 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:33:49.004609 22173 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:33:49.004741 22173 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 11:33:49.004841 22173 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 11:33:49.017278 22173 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.18.212:33883
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786361401922362-8267-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786361401922362-8267-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.8.18.254:39411,127.8.18.253:39493,127.8.18.252:39373
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.253.keytab
--principal=kudu/127.8.18.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.18.253:39493
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786361401922362-8267-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.8.18.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786361401922362-8267-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:33:49.019055 22173 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:33:49.020787 22173 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:33:49.035154 22179 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:33:49.035482 22178 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:33:49.035445 22181 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:33:49.036504 22173 server_base.cc:1034] running on GCE node
I20260810 11:33:49.037519 22173 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:33:49.039198 22173 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:33:49.040480 22173 hybrid_clock.cc:648] HybridClock initialized: now 1786361629040439 us; error 73 us; skew 500 ppm
I20260810 11:33:49.040951 22173 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:33:49 dist-test-slave-srgv krb5kdc[22084](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361629, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:33:49.044831 22173 init.cc:377] Logged in from keytab as kudu/127.8.18.253@KRBTEST.COM (short username kudu)
I20260810 11:33:49.047250 22173 webserver.cc:460] Webserver started at http://127.8.18.253:36537/ using document root <none> and password file <none>
I20260810 11:33:49.048195 22173 fs_manager.cc:359] Metadata directory not provided
I20260810 11:33:49.048373 22173 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:33:49.048760 22173 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:33:49.052531 22173 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786361401922362-8267-0/minicluster-data/master-1/data/instance:
uuid: "af9a61df37a84440b3f67826ff8c4068"
format_stamp: "Formatted at 2026-08-10 11:33:49 on dist-test-slave-srgv"
server_key: "505353c4ff9a6b6b95859c5dda118542"
server_key_iv: "c267a581488d59f1a727c863660778aa"
server_key_version: "encryptionkey@0"
I20260810 11:33:49.053797 22173 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786361401922362-8267-0/minicluster-data/master-1/wal/instance:
uuid: "af9a61df37a84440b3f67826ff8c4068"
format_stamp: "Formatted at 2026-08-10 11:33:49 on dist-test-slave-srgv"
server_key: "505353c4ff9a6b6b95859c5dda118542"
server_key_iv: "c267a581488d59f1a727c863660778aa"
server_key_version: "encryptionkey@0"
I20260810 11:33:49.060436 22173 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260810 11:33:49.065016 22188 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:33:49.067132 22173 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260810 11:33:49.067399 22173 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786361401922362-8267-0/minicluster-data/master-1/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786361401922362-8267-0/minicluster-data/master-1/wal
uuid: "af9a61df37a84440b3f67826ff8c4068"
format_stamp: "Formatted at 2026-08-10 11:33:49 on dist-test-slave-srgv"
server_key: "505353c4ff9a6b6b95859c5dda118542"
server_key_iv: "c267a581488d59f1a727c863660778aa"
server_key_version: "encryptionkey@0"
I20260810 11:33:49.067636 22173 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786361401922362-8267-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786361401922362-8267-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786361401922362-8267-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:33:49.098420 22173 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:33:49.104154 22173 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:33:49.104476 22173 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:33:49.128235 22173 rpc_server.cc:307] RPC server started. Bound to: 127.8.18.253:39493
I20260810 11:33:49.128266 22241 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.18.253:39493 every 8 connection(s)
I20260810 11:33:49.130064 22173 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786361401922362-8267-0/minicluster-data/master-1/data/info.pb
I20260810 11:33:49.133071 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 22173
I20260810 11:33:49.133306 8267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786361401922362-8267-0/minicluster-data/master-1/wal/instance
I20260810 11:33:49.133852 8267 external_mini_cluster.cc:1428] Setting key 7a7979eed5b04141bfafb677f03baf68
I20260810 11:33:49.135092 22242 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:33:49.143034 22242 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.18.254" port: 39411 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.8.18.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.18.252@KRBTEST.COM" created.
Aug 10 11:33:49 dist-test-slave-srgv krb5kdc[22084](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361629, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.253@KRBTEST.COM for kudu/127.8.18.254@KRBTEST.COM
I20260810 11:33:49.175670 22245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:33:49.146597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58352 (local address 127.8.18.254:39411)
0810 11:33:49.147306 (+ 709us) server_negotiation.cc:206] Beginning negotiation
0810 11:33:49.147315 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 11:33:49.148949 (+ 1634us) server_negotiation.cc:407] Connection header received
0810 11:33:49.151641 (+ 2692us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:33:49.151651 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:33:49.151786 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:33:49.151943 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:33:49.154279 (+ 2336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:33:49.155671 (+ 1392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:33:49.158244 (+ 2573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:33:49.159071 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:33:49.159249 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:33:49.166782 (+ 7533us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:33:49.166789 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:33:49.166793 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:33:49.166846 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:33:49.170515 (+ 3669us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:33:49.172132 (+ 1617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:33:49.172138 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:33:49.172140 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:33:49.172306 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:33:49.173024 (+ 718us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:33:49.173033 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:33:49.173040 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:33:49.173599 (+ 559us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:33:49.173825 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0810 11:33:49.174780 (+ 955us) server_negotiation.cc:299] Negotiation successful
0810 11:33:49.175108 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":539,"thread_start_us":370,"threads_started":1}
I20260810 11:33:49.176514 22246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:33:49.147475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:39411 (local address 127.0.0.1:58352)
0810 11:33:49.148712 (+ 1237us) negotiation.cc:107] Waiting for socket to connect
0810 11:33:49.148774 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0810 11:33:49.151181 (+ 2407us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:33:49.152348 (+ 1167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:33:49.152359 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:33:49.153110 (+ 751us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:33:49.153979 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:33:49.154004 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:33:49.155927 (+ 1923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:33:49.155932 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:33:49.157996 (+ 2064us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:33:49.158029 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:33:49.159429 (+ 1400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:33:49.159435 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:33:49.159697 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:33:49.160725 (+ 1028us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:33:49.160755 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:33:49.166349 (+ 5594us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:33:49.170813 (+ 4464us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:33:49.170829 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:33:49.170866 (+ 37us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:33:49.171855 (+ 989us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:33:49.172457 (+ 602us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:33:49.172463 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:33:49.172467 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:33:49.172862 (+ 395us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:33:49.173792 (+ 930us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:33:49.173800 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:33:49.174285 (+ 485us) client_negotiation.cc:769] Sending connection context
0810 11:33:49.174577 (+ 292us) client_negotiation.cc:240] Negotiation successful
0810 11:33:49.175070 (+ 493us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":830,"thread_start_us":477,"threads_started":1}
I20260810 11:33:49.179097 22242 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.18.253" port: 39493 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 11:33:49 dist-test-slave-srgv krb5kdc[22084](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361629, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.253@KRBTEST.COM for kudu/127.8.18.253@KRBTEST.COM
I20260810 11:33:49.197585 22246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:33:49.180084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.253:39493 (local address 127.0.0.1:60450)
0810 11:33:49.180293 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0810 11:33:49.180314 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 11:33:49.180447 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:33:49.181920 (+ 1473us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:33:49.181925 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:33:49.182466 (+ 541us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:33:49.182629 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:33:49.182639 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:33:49.184143 (+ 1504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:33:49.184148 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:33:49.185065 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:33:49.185078 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:33:49.186095 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:33:49.186101 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:33:49.186328 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:33:49.187194 (+ 866us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:33:49.187215 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:33:49.190131 (+ 2916us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:33:49.194571 (+ 4440us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:33:49.194578 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:33:49.194582 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:33:49.195007 (+ 425us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:33:49.195491 (+ 484us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:33:49.195501 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:33:49.195504 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:33:49.195689 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:33:49.196424 (+ 735us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:33:49.196429 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:33:49.196594 (+ 165us) client_negotiation.cc:769] Sending connection context
0810 11:33:49.196757 (+ 163us) client_negotiation.cc:240] Negotiation successful
0810 11:33:49.197110 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":63}
I20260810 11:33:49.197680 22248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:33:49.180434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60450 (local address 127.8.18.253:39493)
0810 11:33:49.181327 (+ 893us) server_negotiation.cc:206] Beginning negotiation
0810 11:33:49.181334 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:33:49.181366 (+ 32us) server_negotiation.cc:407] Connection header received
0810 11:33:49.181491 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:33:49.181501 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:33:49.181743 (+ 242us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:33:49.181936 (+ 193us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:33:49.182809 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:33:49.183954 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:33:49.185237 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:33:49.185923 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:33:49.186080 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:33:49.190435 (+ 4355us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:33:49.190456 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:33:49.190472 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:33:49.190553 (+ 81us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:33:49.194352 (+ 3799us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:33:49.195187 (+ 835us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:33:49.195205 (+ 18us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:33:49.195207 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:33:49.195343 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:33:49.195816 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:33:49.195822 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:33:49.195825 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:33:49.196241 (+ 416us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:33:49.196510 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0810 11:33:49.196929 (+ 419us) server_negotiation.cc:299] Negotiation successful
0810 11:33:49.197244 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":32,"server-negotiator.queue_time_us":713,"thread_start_us":527,"threads_started":1}
I20260810 11:33:49.200639 22242 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.18.252" port: 39373 } has no permanent_uuid. Determining permanent_uuid...
I20260810 11:33:49.202342 22246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:33:49.201591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.252:39373 (local address 127.0.0.1:37772)
0810 11:33:49.201976 (+ 385us) negotiation.cc:107] Waiting for socket to connect
0810 11:33:49.202133 (+ 157us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.252:39373: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":212}
W20260810 11:33:49.202733 22189 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.18.252:39373: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.18.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.252.keytab.
Entry for principal kudu/127.8.18.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.252.keytab.
I20260810 11:33:49.205212 22246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:33:49.204667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.252:39373 (local address 127.0.0.1:37788)
0810 11:33:49.204888 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0810 11:33:49.205008 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.252:39373: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
W20260810 11:33:49.205690 22242 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.18.252:39373: Network error: Client connection negotiation failed: client connection to 127.8.18.252:39373: connect: Connection refused (error 111)
I20260810 11:33:49.206331 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786361401922362-8267-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786361401922362-8267-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786361401922362-8267-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786361401922362-8267-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.18.252:39373
--webserver_interface=127.8.18.252
--webserver_port=0
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.252.keytab
--principal=kudu/127.8.18.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.18.212:33883
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.18.254:39411,127.8.18.253:39493,127.8.18.252:39373
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786361401922362-8267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786361401922362-8267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786361401922362-8267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 11:33:49.234108 22166 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.18.253" port: 39493 } attempt: 5
Aug 10 11:33:49 dist-test-slave-srgv krb5kdc[22084](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361628, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.254@KRBTEST.COM for kudu/127.8.18.253@KRBTEST.COM
I20260810 11:33:49.251415 22169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:33:49.235165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.253:39493 (local address 127.0.0.1:60460)
0810 11:33:49.235399 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0810 11:33:49.235429 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0810 11:33:49.235623 (+ 194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:33:49.236014 (+ 391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:33:49.236022 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:33:49.236483 (+ 461us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:33:49.236694 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:33:49.236704 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:33:49.238293 (+ 1589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:33:49.238298 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:33:49.239386 (+ 1088us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:33:49.239404 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:33:49.240364 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:33:49.240372 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:33:49.240593 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:33:49.241459 (+ 866us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:33:49.241496 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:33:49.244064 (+ 2568us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:33:49.248372 (+ 4308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:33:49.248379 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:33:49.248383 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:33:49.248863 (+ 480us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:33:49.249361 (+ 498us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:33:49.249367 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:33:49.249371 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:33:49.249616 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:33:49.250242 (+ 626us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:33:49.250247 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:33:49.250462 (+ 215us) client_negotiation.cc:769] Sending connection context
0810 11:33:49.250679 (+ 217us) client_negotiation.cc:240] Negotiation successful
0810 11:33:49.251032 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":78}
I20260810 11:33:49.251390 22248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:33:49.235392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60460 (local address 127.8.18.253:39493)
0810 11:33:49.235654 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0810 11:33:49.235662 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:33:49.235690 (+ 28us) server_negotiation.cc:407] Connection header received
0810 11:33:49.235760 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:33:49.235766 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:33:49.235845 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:33:49.235966 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:33:49.236893 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:33:49.238054 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:33:49.239613 (+ 1559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:33:49.240137 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:33:49.240326 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:33:49.244385 (+ 4059us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:33:49.244392 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:33:49.244395 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:33:49.244439 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:33:49.248150 (+ 3711us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:33:49.249052 (+ 902us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:33:49.249058 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:33:49.249060 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:33:49.249216 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:33:49.249761 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:33:49.249771 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:33:49.249773 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:33:49.250100 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:33:49.250282 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0810 11:33:49.250803 (+ 521us) server_negotiation.cc:299] Negotiation successful
0810 11:33:49.251052 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260810 11:33:49.252825 22166 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.18.252" port: 39373 } has no permanent_uuid. Determining permanent_uuid...
I20260810 11:33:49.254254 22169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:33:49.253758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.252:39373 (local address 127.0.0.1:37800)
0810 11:33:49.253960 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0810 11:33:49.254061 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.252:39373: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260810 11:33:49.255084 22242 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.18.252" port: 39373 } attempt: 1
I20260810 11:33:49.255599 22169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:33:49.255088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.252:39373 (local address 127.0.0.1:37802)
0810 11:33:49.255258 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0810 11:33:49.255410 (+ 152us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.252:39373: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260810 11:33:49.255870 22166 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.18.252:39373: Network error: Client connection negotiation failed: client connection to 127.8.18.252:39373: connect: Connection refused (error 111)
I20260810 11:33:49.256250 22246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:33:49.255768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.252:39373 (local address 127.0.0.1:37814)
0810 11:33:49.255974 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0810 11:33:49.256104 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.252:39373: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260810 11:33:49.257529 22246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:33:49.256980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.252:39373 (local address 127.0.0.1:37822)
0810 11:33:49.257172 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0810 11:33:49.257306 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.252:39373: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
W20260810 11:33:49.257854 22242 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.18.252:39373: Network error: Client connection negotiation failed: client connection to 127.8.18.252:39373: connect: Connection refused (error 111)
I20260810 11:33:49.307200 22166 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.18.252" port: 39373 } attempt: 1
I20260810 11:33:49.308650 22169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:33:49.308026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.252:39373 (local address 127.0.0.1:37830)
0810 11:33:49.308290 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0810 11:33:49.308426 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.252:39373: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97}
I20260810 11:33:49.310082 22169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:33:49.309633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.252:39373 (local address 127.0.0.1:37840)
0810 11:33:49.309819 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0810 11:33:49.309919 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.252:39373: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260810 11:33:49.310345 22166 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.18.252:39373: Network error: Client connection negotiation failed: client connection to 127.8.18.252:39373: connect: Connection refused (error 111)
I20260810 11:33:49.326180 22242 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.18.252" port: 39373 } attempt: 2
I20260810 11:33:49.327591 22246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:33:49.327027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.252:39373 (local address 127.0.0.1:37844)
0810 11:33:49.327283 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0810 11:33:49.327398 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.252:39373: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113}
I20260810 11:33:49.329058 22246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:33:49.328572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.252:39373 (local address 127.0.0.1:37852)
0810 11:33:49.328777 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0810 11:33:49.328871 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.252:39373: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
W20260810 11:33:49.329527 22242 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.18.252:39373: Network error: Client connection negotiation failed: client connection to 127.8.18.252:39373: connect: Connection refused (error 111)
I20260810 11:33:49.378647 22166 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.18.252" port: 39373 } attempt: 2
I20260810 11:33:49.380255 22169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:33:49.379549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.252:39373 (local address 127.0.0.1:37862)
0810 11:33:49.379875 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0810 11:33:49.380012 (+ 137us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.252:39373: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95}
I20260810 11:33:49.381618 22169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:33:49.381138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.252:39373 (local address 127.0.0.1:37870)
0810 11:33:49.381329 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0810 11:33:49.381471 (+ 142us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.252:39373: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260810 11:33:49.381951 22166 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.18.252:39373: Network error: Client connection negotiation failed: client connection to 127.8.18.252:39373: connect: Connection refused (error 111)
I20260810 11:33:49.420882 22242 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.18.252" port: 39373 } attempt: 3
I20260810 11:33:49.422441 22246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:33:49.421751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.252:39373 (local address 127.0.0.1:37884)
0810 11:33:49.422064 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0810 11:33:49.422199 (+ 135us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.252:39373: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131}
I20260810 11:33:49.423763 22246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:33:49.423198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.252:39373 (local address 127.0.0.1:37896)
0810 11:33:49.423482 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0810 11:33:49.423582 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.252:39373: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156}
W20260810 11:33:49.424109 22242 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.18.252:39373: Network error: Client connection negotiation failed: client connection to 127.8.18.252:39373: connect: Connection refused (error 111)
I20260810 11:33:49.488245 22166 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.18.252" port: 39373 } attempt: 3
I20260810 11:33:49.489763 22169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:33:49.488992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.252:39373 (local address 127.0.0.1:37902)
0810 11:33:49.489342 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0810 11:33:49.489516 (+ 174us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.252:39373: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139}
I20260810 11:33:49.491324 22169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:33:49.490701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.252:39373 (local address 127.0.0.1:37910)
0810 11:33:49.490990 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0810 11:33:49.491123 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.252:39373: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116}
W20260810 11:33:49.491770 22166 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.18.252:39373: Network error: Client connection negotiation failed: client connection to 127.8.18.252:39373: connect: Connection refused (error 111)
I20260810 11:33:49.567444 22242 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.18.252" port: 39373 } attempt: 4
I20260810 11:33:49.568912 22246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:33:49.568269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.252:39373 (local address 127.0.0.1:37918)
0810 11:33:49.568542 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0810 11:33:49.568677 (+ 135us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.252:39373: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260810 11:33:49.570226 22246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:33:49.569765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.252:39373 (local address 127.0.0.1:37928)
0810 11:33:49.569946 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0810 11:33:49.570060 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.252:39373: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260810 11:33:49.570506 22242 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.18.252:39373: Network error: Client connection negotiation failed: client connection to 127.8.18.252:39373: connect: Connection refused (error 111)
W20260810 11:33:49.605965 22250 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:33:49.606305 22250 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:33:49.606501 22250 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:33:49.614548 22250 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 11:33:49.614704 22250 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:33:49.614799 22250 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:33:49.614912 22250 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 11:33:49.614993 22250 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 11:33:49.626111 22250 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.18.212:33883
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786361401922362-8267-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786361401922362-8267-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.8.18.254:39411,127.8.18.253:39493,127.8.18.252:39373
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.252.keytab
--principal=kudu/127.8.18.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.18.252:39373
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786361401922362-8267-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.8.18.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786361401922362-8267-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:33:49.627650 22250 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:33:49.629139 22250 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:33:49.642620 22258 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:33:49.642666 22256 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:33:49.642544 22255 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:33:49.643962 22250 server_base.cc:1034] running on GCE node
I20260810 11:33:49.644942 22250 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:33:49.646646 22250 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:33:49.647890 22250 hybrid_clock.cc:648] HybridClock initialized: now 1786361629647845 us; error 38 us; skew 500 ppm
I20260810 11:33:49.648371 22250 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:33:49 dist-test-slave-srgv krb5kdc[22084](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361629, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:33:49.652097 22250 init.cc:377] Logged in from keytab as kudu/127.8.18.252@KRBTEST.COM (short username kudu)
I20260810 11:33:49.654012 22250 webserver.cc:460] Webserver started at http://127.8.18.252:34837/ using document root <none> and password file <none>
I20260810 11:33:49.654811 22250 fs_manager.cc:359] Metadata directory not provided
I20260810 11:33:49.654925 22250 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:33:49.655292 22250 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:33:49.658876 22250 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786361401922362-8267-0/minicluster-data/master-2/data/instance:
uuid: "e21be99a896b4cce87858590700f425b"
format_stamp: "Formatted at 2026-08-10 11:33:49 on dist-test-slave-srgv"
server_key: "701df408ed47284b66d9bdde3c11710f"
server_key_iv: "e7efdf4d3b71956a4ed2bdab3a016fed"
server_key_version: "encryptionkey@0"
I20260810 11:33:49.659826 22250 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786361401922362-8267-0/minicluster-data/master-2/wal/instance:
uuid: "e21be99a896b4cce87858590700f425b"
format_stamp: "Formatted at 2026-08-10 11:33:49 on dist-test-slave-srgv"
server_key: "701df408ed47284b66d9bdde3c11710f"
server_key_iv: "e7efdf4d3b71956a4ed2bdab3a016fed"
server_key_version: "encryptionkey@0"
I20260810 11:33:49.666164 22250 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260810 11:33:49.669111 22166 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.18.252" port: 39373 } attempt: 4
I20260810 11:33:49.670347 22169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:33:49.669797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.252:39373 (local address 127.0.0.1:37934)
0810 11:33:49.670072 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0810 11:33:49.670185 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.252:39373: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260810 11:33:49.670431 22265 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:33:49.671765 22169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:33:49.671279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.252:39373 (local address 127.0.0.1:37950)
0810 11:33:49.671474 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0810 11:33:49.671558 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.252:39373: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260810 11:33:49.672165 22166 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.18.252:39373: Network error: Client connection negotiation failed: client connection to 127.8.18.252:39373: connect: Connection refused (error 111)
I20260810 11:33:49.672492 22250 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260810 11:33:49.672703 22250 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786361401922362-8267-0/minicluster-data/master-2/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786361401922362-8267-0/minicluster-data/master-2/wal
uuid: "e21be99a896b4cce87858590700f425b"
format_stamp: "Formatted at 2026-08-10 11:33:49 on dist-test-slave-srgv"
server_key: "701df408ed47284b66d9bdde3c11710f"
server_key_iv: "e7efdf4d3b71956a4ed2bdab3a016fed"
server_key_version: "encryptionkey@0"
I20260810 11:33:49.672935 22250 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786361401922362-8267-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786361401922362-8267-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786361401922362-8267-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:33:49.726186 22250 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:33:49.731065 22250 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:33:49.731498 22250 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:33:49.754030 22250 rpc_server.cc:307] RPC server started. Bound to: 127.8.18.252:39373
I20260810 11:33:49.754156 22316 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.18.252:39373 every 8 connection(s)
I20260810 11:33:49.755735 22250 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786361401922362-8267-0/minicluster-data/master-2/data/info.pb
I20260810 11:33:49.759773 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 22250
I20260810 11:33:49.759992 8267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786361401922362-8267-0/minicluster-data/master-2/wal/instance
I20260810 11:33:49.760490 22317 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:33:49.760669 8267 external_mini_cluster.cc:1428] Setting key 5a37de22c76d02614cf397f4163b5b25
I20260810 11:33:49.771438 22317 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.18.254" port: 39411 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 11:33:49 dist-test-slave-srgv krb5kdc[22084](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361627, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.18.252@KRBTEST.COM
Aug 10 11:33:49 dist-test-slave-srgv krb5kdc[22084](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361629, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.252@KRBTEST.COM for kudu/127.8.18.254@KRBTEST.COM
I20260810 11:33:49.788384 22320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:33:49.763068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37960 (local address 127.8.18.252:39373)
0810 11:33:49.764022 (+ 954us) server_negotiation.cc:206] Beginning negotiation
0810 11:33:49.764037 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 11:33:49.764137 (+ 100us) server_negotiation.cc:407] Connection header received
0810 11:33:49.766342 (+ 2205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:33:49.766373 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:33:49.766990 (+ 617us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:33:49.767491 (+ 501us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:33:49.769266 (+ 1775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:33:49.770825 (+ 1559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:33:49.772564 (+ 1739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:33:49.773527 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:33:49.773768 (+ 241us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:33:49.780023 (+ 6255us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:33:49.780033 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:33:49.780046 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:33:49.780100 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:33:49.783625 (+ 3525us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:33:49.784631 (+ 1006us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:33:49.784649 (+ 18us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:33:49.784656 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:33:49.784841 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:33:49.785271 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:33:49.785277 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:33:49.785280 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:33:49.785936 (+ 656us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:33:49.786294 (+ 358us) server_negotiation.cc:1035] Waiting for connection context
0810 11:33:49.786811 (+ 517us) server_negotiation.cc:299] Negotiation successful
0810 11:33:49.787305 (+ 494us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":692,"thread_start_us":447,"threads_started":1}
I20260810 11:33:49.791538 22322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:33:49.774901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:39411 (local address 127.0.0.1:58354)
0810 11:33:49.775820 (+ 919us) negotiation.cc:107] Waiting for socket to connect
0810 11:33:49.775871 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0810 11:33:49.776056 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:33:49.776607 (+ 551us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:33:49.776613 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:33:49.777527 (+ 914us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:33:49.777882 (+ 355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:33:49.777894 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:33:49.779173 (+ 1279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:33:49.779178 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:33:49.780323 (+ 1145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:33:49.780344 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:33:49.781464 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:33:49.781471 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:33:49.781684 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:33:49.782410 (+ 726us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:33:49.782430 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:33:49.784875 (+ 2445us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:33:49.788426 (+ 3551us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:33:49.788440 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:33:49.788450 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:33:49.789007 (+ 557us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:33:49.789507 (+ 500us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:33:49.789513 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:33:49.789517 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:33:49.789690 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:33:49.790303 (+ 613us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:33:49.790311 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:33:49.790503 (+ 192us) client_negotiation.cc:769] Sending connection context
0810 11:33:49.790717 (+ 214us) client_negotiation.cc:240] Negotiation successful
0810 11:33:49.791110 (+ 393us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":619,"thread_start_us":461,"threads_started":1}
I20260810 11:33:49.791523 22321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:33:49.774877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58354 (local address 127.8.18.254:39411)
0810 11:33:49.775470 (+ 593us) server_negotiation.cc:206] Beginning negotiation
0810 11:33:49.775479 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 11:33:49.775999 (+ 520us) server_negotiation.cc:407] Connection header received
0810 11:33:49.776208 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:33:49.776214 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:33:49.776335 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:33:49.776475 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:33:49.778087 (+ 1612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:33:49.778982 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:33:49.780513 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:33:49.781186 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:33:49.781383 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:33:49.785123 (+ 3740us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:33:49.785128 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:33:49.785132 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:33:49.785169 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:33:49.788271 (+ 3102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:33:49.789166 (+ 895us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:33:49.789175 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:33:49.789181 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:33:49.789341 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:33:49.789832 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:33:49.789838 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:33:49.789842 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:33:49.790144 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:33:49.790321 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0810 11:33:49.790852 (+ 531us) server_negotiation.cc:299] Negotiation successful
0810 11:33:49.791095 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":464,"thread_start_us":293,"threads_started":1}
I20260810 11:33:49.793262 22317 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.18.253" port: 39493 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 11:33:49 dist-test-slave-srgv krb5kdc[22084](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361629, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.252@KRBTEST.COM for kudu/127.8.18.253@KRBTEST.COM
Aug 10 11:33:49 dist-test-slave-srgv krb5kdc[22084](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361627, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.18.254@KRBTEST.COM
I20260810 11:33:49.811518 22322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:33:49.794372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.253:39493 (local address 127.0.0.1:60476)
0810 11:33:49.794565 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0810 11:33:49.794588 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0810 11:33:49.794698 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:33:49.795454 (+ 756us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:33:49.795460 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:33:49.796014 (+ 554us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:33:49.796172 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:33:49.796181 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:33:49.797940 (+ 1759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:33:49.797944 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:33:49.799349 (+ 1405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:33:49.799364 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:33:49.800513 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:33:49.800518 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:33:49.800701 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:33:49.801346 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:33:49.801366 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:33:49.803892 (+ 2526us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:33:49.808379 (+ 4487us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:33:49.808389 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:33:49.808394 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:33:49.808921 (+ 527us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:33:49.809489 (+ 568us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:33:49.809497 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:33:49.809502 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:33:49.809713 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:33:49.810384 (+ 671us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:33:49.810389 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:33:49.810591 (+ 202us) client_negotiation.cc:769] Sending connection context
0810 11:33:49.810762 (+ 171us) client_negotiation.cc:240] Negotiation successful
0810 11:33:49.811141 (+ 379us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":65}
I20260810 11:33:49.811555 22321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:33:49.792373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58366 (local address 127.8.18.254:39411)
0810 11:33:49.792594 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0810 11:33:49.792600 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:33:49.792618 (+ 18us) server_negotiation.cc:407] Connection header received
0810 11:33:49.792665 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:33:49.792669 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:33:49.792753 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:33:49.792865 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:33:49.794731 (+ 1866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:33:49.795794 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:33:49.798171 (+ 2377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:33:49.798868 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:33:49.799012 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:33:49.805216 (+ 6204us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:33:49.805224 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:33:49.805227 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:33:49.805300 (+ 73us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:33:49.808233 (+ 2933us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:33:49.809211 (+ 978us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:33:49.809217 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:33:49.809219 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:33:49.809373 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:33:49.809923 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:33:49.809927 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:33:49.809929 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:33:49.810242 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:33:49.810584 (+ 342us) server_negotiation.cc:1035] Waiting for connection context
0810 11:33:49.810975 (+ 391us) server_negotiation.cc:299] Negotiation successful
0810 11:33:49.811233 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260810 11:33:49.811718 22323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:33:49.794334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60476 (local address 127.8.18.253:39493)
0810 11:33:49.795120 (+ 786us) server_negotiation.cc:206] Beginning negotiation
0810 11:33:49.795125 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:33:49.795144 (+ 19us) server_negotiation.cc:407] Connection header received
0810 11:33:49.795197 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:33:49.795201 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:33:49.795270 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:33:49.795379 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:33:49.796366 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:33:49.797760 (+ 1394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:33:49.799545 (+ 1785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:33:49.800292 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:33:49.800468 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:33:49.804266 (+ 3798us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:33:49.804279 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:33:49.804287 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:33:49.804362 (+ 75us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:33:49.808157 (+ 3795us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:33:49.809103 (+ 946us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:33:49.809113 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:33:49.809119 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:33:49.809319 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:33:49.809872 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:33:49.809878 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:33:49.809883 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:33:49.810223 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:33:49.810392 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0810 11:33:49.810904 (+ 512us) server_negotiation.cc:299] Negotiation successful
0810 11:33:49.811198 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":677,"thread_start_us":575,"threads_started":1}
I20260810 11:33:49.813666 22317 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.18.252" port: 39373 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 11:33:49 dist-test-slave-srgv krb5kdc[22084](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361629, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.252@KRBTEST.COM for kudu/127.8.18.252@KRBTEST.COM
Aug 10 11:33:49 dist-test-slave-srgv krb5kdc[22084](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361627, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.18.253@KRBTEST.COM
I20260810 11:33:49.828961 22322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:33:49.814532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.252:39373 (local address 127.0.0.1:37962)
0810 11:33:49.814737 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0810 11:33:49.814775 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0810 11:33:49.814877 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:33:49.815168 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:33:49.815172 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:33:49.815614 (+ 442us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:33:49.815757 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:33:49.815766 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:33:49.816774 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:33:49.816781 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:33:49.817781 (+ 1000us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:33:49.817794 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:33:49.818538 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:33:49.818544 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:33:49.818755 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:33:49.819427 (+ 672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:33:49.819445 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:33:49.822199 (+ 2754us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:33:49.826069 (+ 3870us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:33:49.826082 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:33:49.826085 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:33:49.826685 (+ 600us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:33:49.827140 (+ 455us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:33:49.827145 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:33:49.827148 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:33:49.827333 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:33:49.827975 (+ 642us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:33:49.827980 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:33:49.828196 (+ 216us) client_negotiation.cc:769] Sending connection context
0810 11:33:49.828329 (+ 133us) client_negotiation.cc:240] Negotiation successful
0810 11:33:49.828622 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":68}
I20260810 11:33:49.829525 22320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:33:49.814711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37962 (local address 127.8.18.252:39373)
0810 11:33:49.814869 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0810 11:33:49.814874 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:33:49.814891 (+ 17us) server_negotiation.cc:407] Connection header received
0810 11:33:49.814973 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:33:49.814977 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:33:49.815042 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:33:49.815150 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:33:49.815904 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:33:49.816641 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:33:49.817923 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:33:49.818365 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:33:49.818515 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:33:49.822471 (+ 3956us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:33:49.822480 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:33:49.822487 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:33:49.822541 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:33:49.825896 (+ 3355us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:33:49.826833 (+ 937us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:33:49.826838 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:33:49.826840 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:33:49.826977 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:33:49.827483 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:33:49.827487 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:33:49.827489 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:33:49.827839 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:33:49.828019 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0810 11:33:49.828466 (+ 447us) server_negotiation.cc:299] Negotiation successful
0810 11:33:49.828870 (+ 404us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":77,"server-negotiator.queue_time_us":58}
I20260810 11:33:49.831516 22323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:33:49.813830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60486 (local address 127.8.18.253:39493)
0810 11:33:49.814001 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0810 11:33:49.814007 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:33:49.814134 (+ 127us) server_negotiation.cc:407] Connection header received
0810 11:33:49.814298 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:33:49.814305 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:33:49.814406 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:33:49.814507 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:33:49.816205 (+ 1698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:33:49.816967 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:33:49.818389 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:33:49.818851 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:33:49.819003 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:33:49.824354 (+ 5351us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:33:49.824360 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:33:49.824364 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:33:49.824403 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:33:49.827845 (+ 3442us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:33:49.828975 (+ 1130us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:33:49.828984 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:33:49.828990 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:33:49.829176 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:33:49.829758 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:33:49.829766 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:33:49.829771 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:33:49.830098 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:33:49.830450 (+ 352us) server_negotiation.cc:1035] Waiting for connection context
0810 11:33:49.830801 (+ 351us) server_negotiation.cc:299] Negotiation successful
0810 11:33:49.831133 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260810 11:33:49.836527 22317 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e21be99a896b4cce87858590700f425b: Bootstrap starting.
I20260810 11:33:49.837934 8267 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 11:33:49.840392 22317 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e21be99a896b4cce87858590700f425b: Neither blocks nor log segments found. Creating new log.
I20260810 11:33:49.841826 22317 log.cc:824] T 00000000000000000000000000000000 P e21be99a896b4cce87858590700f425b: Log is configured to *not* fsync() on all Append() calls
I20260810 11:33:49.846499 22317 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e21be99a896b4cce87858590700f425b: No bootstrap required, opened a new log
I20260810 11:33:49.859723 22317 raft_consensus.cc:348] T 00000000000000000000000000000000 P e21be99a896b4cce87858590700f425b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "837a7216db5341ceb09244a6dd0b6c98" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 39411 } } peers { permanent_uuid: "af9a61df37a84440b3f67826ff8c4068" member_type: VOTER last_known_addr { host: "127.8.18.253" port: 39493 } } peers { permanent_uuid: "e21be99a896b4cce87858590700f425b" member_type: VOTER last_known_addr { host: "127.8.18.252" port: 39373 } }
I20260810 11:33:49.860532 22317 raft_consensus.cc:374] T 00000000000000000000000000000000 P e21be99a896b4cce87858590700f425b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:33:49.860718 22317 raft_consensus.cc:729] T 00000000000000000000000000000000 P e21be99a896b4cce87858590700f425b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e21be99a896b4cce87858590700f425b, State: Initialized, Role: FOLLOWER
I20260810 11:33:49.861743 22317 consensus_queue.cc:260] T 00000000000000000000000000000000 P e21be99a896b4cce87858590700f425b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "837a7216db5341ceb09244a6dd0b6c98" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 39411 } } peers { permanent_uuid: "af9a61df37a84440b3f67826ff8c4068" member_type: VOTER last_known_addr { host: "127.8.18.253" port: 39493 } } peers { permanent_uuid: "e21be99a896b4cce87858590700f425b" member_type: VOTER last_known_addr { host: "127.8.18.252" port: 39373 } }
I20260810 11:33:49.868888 22323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:33:49.843800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60502 (local address 127.8.18.253:39493)
0810 11:33:49.844033 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0810 11:33:49.844061 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0810 11:33:49.845190 (+ 1129us) server_negotiation.cc:407] Connection header received
0810 11:33:49.845531 (+ 341us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:33:49.845536 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:33:49.845623 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:33:49.845713 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:33:49.848792 (+ 3079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:33:49.850200 (+ 1408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:33:49.853222 (+ 3022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:33:49.853987 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:33:49.854112 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:33:49.859446 (+ 5334us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:33:49.859454 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:33:49.859459 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:33:49.859514 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:33:49.863945 (+ 4431us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:33:49.865526 (+ 1581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:33:49.865534 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:33:49.865538 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:33:49.865838 (+ 300us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:33:49.866444 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:33:49.866454 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:33:49.866457 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:33:49.867000 (+ 543us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:33:49.867316 (+ 316us) server_negotiation.cc:1035] Waiting for connection context
0810 11:33:49.868156 (+ 840us) server_negotiation.cc:299] Negotiation successful
0810 11:33:49.868479 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260810 11:33:49.868888 22321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:33:49.844255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58368 (local address 127.8.18.254:39411)
0810 11:33:49.844474 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0810 11:33:49.844481 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:33:49.844887 (+ 406us) server_negotiation.cc:407] Connection header received
0810 11:33:49.845237 (+ 350us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:33:49.845249 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:33:49.845353 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:33:49.845484 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:33:49.849174 (+ 3690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:33:49.850853 (+ 1679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:33:49.853645 (+ 2792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:33:49.854434 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:33:49.854546 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:33:49.859453 (+ 4907us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:33:49.859461 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:33:49.859465 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:33:49.859514 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:33:49.864607 (+ 5093us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:33:49.865775 (+ 1168us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:33:49.865781 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:33:49.865785 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:33:49.865978 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:33:49.866483 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:33:49.866488 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:33:49.866491 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:33:49.866968 (+ 477us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:33:49.867239 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0810 11:33:49.868189 (+ 950us) server_negotiation.cc:299] Negotiation successful
0810 11:33:49.868472 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260810 11:33:49.869293 22333 sys_catalog.cc:455] T 00000000000000000000000000000000 P e21be99a896b4cce87858590700f425b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "837a7216db5341ceb09244a6dd0b6c98" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 39411 } } peers { permanent_uuid: "af9a61df37a84440b3f67826ff8c4068" member_type: VOTER last_known_addr { host: "127.8.18.253" port: 39493 } } peers { permanent_uuid: "e21be99a896b4cce87858590700f425b" member_type: VOTER last_known_addr { host: "127.8.18.252" port: 39373 } } }
I20260810 11:33:49.869726 22242 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.18.252" port: 39373 } attempt: 5
I20260810 11:33:49.873984 22333 sys_catalog.cc:458] T 00000000000000000000000000000000 P e21be99a896b4cce87858590700f425b [sys.catalog]: This master's current role is: FOLLOWER
I20260810 11:33:49.873780 22317 sys_catalog.cc:564] T 00000000000000000000000000000000 P e21be99a896b4cce87858590700f425b [sys.catalog]: configured and running, proceeding with master startup.
I20260810 11:33:49.876415 22320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:33:49.845631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37966 (local address 127.8.18.252:39373)
0810 11:33:49.845827 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0810 11:33:49.845834 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:33:49.853063 (+ 7229us) server_negotiation.cc:407] Connection header received
0810 11:33:49.853401 (+ 338us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:33:49.853437 (+ 36us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:33:49.853558 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:33:49.853648 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:33:49.856376 (+ 2728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:33:49.857640 (+ 1264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:33:49.859998 (+ 2358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:33:49.860926 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:33:49.861089 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:33:49.865996 (+ 4907us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:33:49.866008 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:33:49.866016 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:33:49.866082 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:33:49.871974 (+ 5892us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:33:49.873234 (+ 1260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:33:49.873241 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:33:49.873245 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:33:49.873487 (+ 242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:33:49.874429 (+ 942us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:33:49.874437 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:33:49.874444 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:33:49.874908 (+ 464us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:33:49.875242 (+ 334us) server_negotiation.cc:1035] Waiting for connection context
0810 11:33:49.875664 (+ 422us) server_negotiation.cc:299] Negotiation successful
0810 11:33:49.875946 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
Aug 10 11:33:49 dist-test-slave-srgv krb5kdc[22084](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361629, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.253@KRBTEST.COM for kudu/127.8.18.252@KRBTEST.COM
I20260810 11:33:49.884457 22333 raft_consensus.cc:482] T 00000000000000000000000000000000 P e21be99a896b4cce87858590700f425b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 11:33:49.884804 22333 raft_consensus.cc:504] T 00000000000000000000000000000000 P e21be99a896b4cce87858590700f425b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "837a7216db5341ceb09244a6dd0b6c98" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 39411 } } peers { permanent_uuid: "af9a61df37a84440b3f67826ff8c4068" member_type: VOTER last_known_addr { host: "127.8.18.253" port: 39493 } } peers { permanent_uuid: "e21be99a896b4cce87858590700f425b" member_type: VOTER last_known_addr { host: "127.8.18.252" port: 39373 } }
I20260810 11:33:49.886636 22333 leader_election.cc:288] T 00000000000000000000000000000000 P e21be99a896b4cce87858590700f425b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 837a7216db5341ceb09244a6dd0b6c98 (127.8.18.254:39411), af9a61df37a84440b3f67826ff8c4068 (127.8.18.253:39493)
I20260810 11:33:49.892958 22246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:33:49.870420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.252:39373 (local address 127.0.0.1:37982)
0810 11:33:49.870649 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0810 11:33:49.870671 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 11:33:49.870798 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:33:49.872241 (+ 1443us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:33:49.872246 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:33:49.872869 (+ 623us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:33:49.873139 (+ 270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:33:49.873155 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:33:49.875384 (+ 2229us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:33:49.875391 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:33:49.877383 (+ 1992us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:33:49.877453 (+ 70us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:33:49.878964 (+ 1511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:33:49.878969 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:33:49.879172 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:33:49.880026 (+ 854us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:33:49.880054 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:33:49.882998 (+ 2944us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:33:49.889548 (+ 6550us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:33:49.889555 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:33:49.889560 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:33:49.890263 (+ 703us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:33:49.890840 (+ 577us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:33:49.890846 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:33:49.890850 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:33:49.891050 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:33:49.891798 (+ 748us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:33:49.891802 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:33:49.891978 (+ 176us) client_negotiation.cc:769] Sending connection context
0810 11:33:49.892157 (+ 179us) client_negotiation.cc:240] Negotiation successful
0810 11:33:49.892510 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":84}
I20260810 11:33:49.893002 22334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:33:49.870844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37982 (local address 127.8.18.252:39373)
0810 11:33:49.871800 (+ 956us) server_negotiation.cc:206] Beginning negotiation
0810 11:33:49.871807 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:33:49.871835 (+ 28us) server_negotiation.cc:407] Connection header received
0810 11:33:49.871911 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:33:49.871915 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:33:49.872004 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:33:49.873359 (+ 1355us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:33:49.873736 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:33:49.875157 (+ 1421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:33:49.877703 (+ 2546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:33:49.878712 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:33:49.878902 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:33:49.883425 (+ 4523us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:33:49.883435 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:33:49.883440 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:33:49.883494 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:33:49.889298 (+ 5804us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:33:49.890479 (+ 1181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:33:49.890485 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:33:49.890488 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:33:49.890679 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:33:49.891213 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:33:49.891220 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:33:49.891226 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:33:49.891663 (+ 437us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:33:49.891847 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0810 11:33:49.892280 (+ 433us) server_negotiation.cc:299] Negotiation successful
0810 11:33:49.892540 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":753,"thread_start_us":464,"threads_started":1}
I20260810 11:33:49.899860 22242 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P af9a61df37a84440b3f67826ff8c4068: Bootstrap starting.
I20260810 11:33:49.904517 22242 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P af9a61df37a84440b3f67826ff8c4068: Neither blocks nor log segments found. Creating new log.
I20260810 11:33:49.906349 22242 log.cc:824] T 00000000000000000000000000000000 P af9a61df37a84440b3f67826ff8c4068: Log is configured to *not* fsync() on all Append() calls
I20260810 11:33:49.910501 22242 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P af9a61df37a84440b3f67826ff8c4068: No bootstrap required, opened a new log
W20260810 11:33:49.916255 22345 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e21be99a896b4cce87858590700f425b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 11:33:49.916567 22345 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 11:33:49.902113 22250 master_runner.cc:186] Error getting master registration for 127.8.18.254:39411: OK, instance_id { permanent_uuid: "837a7216db5341ceb09244a6dd0b6c98" instance_seqno: 1786361628508925 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260810 11:33:49.920943 22250 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260810 11:33:49.923148 22242 raft_consensus.cc:348] T 00000000000000000000000000000000 P af9a61df37a84440b3f67826ff8c4068 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "837a7216db5341ceb09244a6dd0b6c98" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 39411 } } peers { permanent_uuid: "af9a61df37a84440b3f67826ff8c4068" member_type: VOTER last_known_addr { host: "127.8.18.253" port: 39493 } } peers { permanent_uuid: "e21be99a896b4cce87858590700f425b" member_type: VOTER last_known_addr { host: "127.8.18.252" port: 39373 } }
I20260810 11:33:49.923677 22242 raft_consensus.cc:374] T 00000000000000000000000000000000 P af9a61df37a84440b3f67826ff8c4068 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:33:49.923820 22242 raft_consensus.cc:729] T 00000000000000000000000000000000 P af9a61df37a84440b3f67826ff8c4068 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af9a61df37a84440b3f67826ff8c4068, State: Initialized, Role: FOLLOWER
I20260810 11:33:49.924680 22242 consensus_queue.cc:260] T 00000000000000000000000000000000 P af9a61df37a84440b3f67826ff8c4068 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "837a7216db5341ceb09244a6dd0b6c98" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 39411 } } peers { permanent_uuid: "af9a61df37a84440b3f67826ff8c4068" member_type: VOTER last_known_addr { host: "127.8.18.253" port: 39493 } } peers { permanent_uuid: "e21be99a896b4cce87858590700f425b" member_type: VOTER last_known_addr { host: "127.8.18.252" port: 39373 } }
I20260810 11:33:49.925948 22348 sys_catalog.cc:455] T 00000000000000000000000000000000 P af9a61df37a84440b3f67826ff8c4068 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "837a7216db5341ceb09244a6dd0b6c98" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 39411 } } peers { permanent_uuid: "af9a61df37a84440b3f67826ff8c4068" member_type: VOTER last_known_addr { host: "127.8.18.253" port: 39493 } } peers { permanent_uuid: "e21be99a896b4cce87858590700f425b" member_type: VOTER last_known_addr { host: "127.8.18.252" port: 39373 } } }
I20260810 11:33:49.926460 22348 sys_catalog.cc:458] T 00000000000000000000000000000000 P af9a61df37a84440b3f67826ff8c4068 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 11:33:49.927487 22242 sys_catalog.cc:564] T 00000000000000000000000000000000 P af9a61df37a84440b3f67826ff8c4068 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 11:33:49.928390 22215 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e21be99a896b4cce87858590700f425b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af9a61df37a84440b3f67826ff8c4068" is_pre_election: true
I20260810 11:33:49.929127 22215 raft_consensus.cc:2454] T 00000000000000000000000000000000 P af9a61df37a84440b3f67826ff8c4068 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e21be99a896b4cce87858590700f425b in term 0.
I20260810 11:33:49.929976 22269 leader_election.cc:302] T 00000000000000000000000000000000 P e21be99a896b4cce87858590700f425b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: af9a61df37a84440b3f67826ff8c4068, e21be99a896b4cce87858590700f425b; no voters:
I20260810 11:33:49.930490 22333 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e21be99a896b4cce87858590700f425b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 11:33:49.930675 22333 raft_consensus.cc:482] T 00000000000000000000000000000000 P e21be99a896b4cce87858590700f425b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 11:33:49.930822 22333 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e21be99a896b4cce87858590700f425b [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:33:49.933740 22333 raft_consensus.cc:504] T 00000000000000000000000000000000 P e21be99a896b4cce87858590700f425b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "837a7216db5341ceb09244a6dd0b6c98" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 39411 } } peers { permanent_uuid: "af9a61df37a84440b3f67826ff8c4068" member_type: VOTER last_known_addr { host: "127.8.18.253" port: 39493 } } peers { permanent_uuid: "e21be99a896b4cce87858590700f425b" member_type: VOTER last_known_addr { host: "127.8.18.252" port: 39373 } }
I20260810 11:33:49.934553 22333 leader_election.cc:288] T 00000000000000000000000000000000 P e21be99a896b4cce87858590700f425b [CANDIDATE]: Term 1 election: Requested vote from peers 837a7216db5341ceb09244a6dd0b6c98 (127.8.18.254:39411), af9a61df37a84440b3f67826ff8c4068 (127.8.18.253:39493)
I20260810 11:33:49.935547 22215 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e21be99a896b4cce87858590700f425b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af9a61df37a84440b3f67826ff8c4068"
I20260810 11:33:49.935823 22215 raft_consensus.cc:3037] T 00000000000000000000000000000000 P af9a61df37a84440b3f67826ff8c4068 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:33:49.939191 22215 raft_consensus.cc:2454] T 00000000000000000000000000000000 P af9a61df37a84440b3f67826ff8c4068 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e21be99a896b4cce87858590700f425b in term 1.
I20260810 11:33:49.940124 22269 leader_election.cc:302] T 00000000000000000000000000000000 P e21be99a896b4cce87858590700f425b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: af9a61df37a84440b3f67826ff8c4068, e21be99a896b4cce87858590700f425b; no voters:
I20260810 11:33:49.940558 22333 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e21be99a896b4cce87858590700f425b [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:33:49.948246 22333 raft_consensus.cc:686] T 00000000000000000000000000000000 P e21be99a896b4cce87858590700f425b [term 1 LEADER]: Becoming Leader. State: Replica: e21be99a896b4cce87858590700f425b, State: Running, Role: LEADER
I20260810 11:33:49.949424 22166 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.18.252" port: 39373 } attempt: 5
I20260810 11:33:49.949271 22333 consensus_queue.cc:237] T 00000000000000000000000000000000 P e21be99a896b4cce87858590700f425b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "837a7216db5341ceb09244a6dd0b6c98" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 39411 } } peers { permanent_uuid: "af9a61df37a84440b3f67826ff8c4068" member_type: VOTER last_known_addr { host: "127.8.18.253" port: 39493 } } peers { permanent_uuid: "e21be99a896b4cce87858590700f425b" member_type: VOTER last_known_addr { host: "127.8.18.252" port: 39373 } }
Aug 10 11:33:49 dist-test-slave-srgv krb5kdc[22084](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361628, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.254@KRBTEST.COM for kudu/127.8.18.252@KRBTEST.COM
I20260810 11:33:49.969229 22353 sys_catalog.cc:455] T 00000000000000000000000000000000 P e21be99a896b4cce87858590700f425b [sys.catalog]: SysCatalogTable state changed. Reason: New leader e21be99a896b4cce87858590700f425b. Latest consensus state: current_term: 1 leader_uuid: "e21be99a896b4cce87858590700f425b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "837a7216db5341ceb09244a6dd0b6c98" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 39411 } } peers { permanent_uuid: "af9a61df37a84440b3f67826ff8c4068" member_type: VOTER last_known_addr { host: "127.8.18.253" port: 39493 } } peers { permanent_uuid: "e21be99a896b4cce87858590700f425b" member_type: VOTER last_known_addr { host: "127.8.18.252" port: 39373 } } }
I20260810 11:33:49.969832 22353 sys_catalog.cc:458] T 00000000000000000000000000000000 P e21be99a896b4cce87858590700f425b [sys.catalog]: This master's current role is: LEADER
I20260810 11:33:49.979131 22360 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
W20260810 11:33:49.984278 22271 tablet.cc:2369] T 00000000000000000000000000000000 P e21be99a896b4cce87858590700f425b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 11:33:49.986196 22169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:33:49.950159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.252:39373 (local address 127.0.0.1:37990)
0810 11:33:49.950583 (+ 424us) negotiation.cc:107] Waiting for socket to connect
0810 11:33:49.950606 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0810 11:33:49.951049 (+ 443us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:33:49.951605 (+ 556us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:33:49.951613 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:33:49.952167 (+ 554us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:33:49.952429 (+ 262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:33:49.952451 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:33:49.954298 (+ 1847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:33:49.954307 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:33:49.956486 (+ 2179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:33:49.956515 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:33:49.958817 (+ 2302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:33:49.958829 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:33:49.959459 (+ 630us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:33:49.961078 (+ 1619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:33:49.961142 (+ 64us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:33:49.965774 (+ 4632us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:33:49.977818 (+ 12044us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:33:49.977825 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:33:49.977832 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:33:49.978559 (+ 727us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:33:49.981366 (+ 2807us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:33:49.981373 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:33:49.981376 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:33:49.981766 (+ 390us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:33:49.983131 (+ 1365us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:33:49.983137 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:33:49.983493 (+ 356us) client_negotiation.cc:769] Sending connection context
0810 11:33:49.985273 (+ 1780us) client_negotiation.cc:240] Negotiation successful
0810 11:33:49.985701 (+ 428us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277}
I20260810 11:33:49.987617 22334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:33:49.950890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37990 (local address 127.8.18.252:39373)
0810 11:33:49.951099 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0810 11:33:49.951107 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:33:49.951134 (+ 27us) server_negotiation.cc:407] Connection header received
0810 11:33:49.951214 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:33:49.951224 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:33:49.951353 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:33:49.951508 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:33:49.952845 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:33:49.954023 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:33:49.956850 (+ 2827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:33:49.958122 (+ 1272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:33:49.958328 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:33:49.966098 (+ 7770us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:33:49.966109 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:33:49.966114 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:33:49.966169 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:33:49.977464 (+ 11295us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:33:49.980911 (+ 3447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:33:49.980918 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:33:49.980921 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:33:49.981154 (+ 233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:33:49.982090 (+ 936us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:33:49.982125 (+ 35us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:33:49.982131 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:33:49.982862 (+ 731us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:33:49.986668 (+ 3806us) server_negotiation.cc:1035] Waiting for connection context
0810 11:33:49.986856 (+ 188us) server_negotiation.cc:299] Negotiation successful
0810 11:33:49.987196 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260810 11:33:49.989323 22360 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 11:33:50.005671 22166 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 837a7216db5341ceb09244a6dd0b6c98: Bootstrap starting.
I20260810 11:33:50.010655 22166 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 837a7216db5341ceb09244a6dd0b6c98: Neither blocks nor log segments found. Creating new log.
I20260810 11:33:50.012811 22166 log.cc:824] T 00000000000000000000000000000000 P 837a7216db5341ceb09244a6dd0b6c98: Log is configured to *not* fsync() on all Append() calls
I20260810 11:33:50.014372 22215 raft_consensus.cc:1261] T 00000000000000000000000000000000 P af9a61df37a84440b3f67826ff8c4068 [term 1 FOLLOWER]: Refusing update from remote peer e21be99a896b4cce87858590700f425b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 11:33:50.018567 22353 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e21be99a896b4cce87858590700f425b [LEADER]: Connected to new peer: Peer: permanent_uuid: "af9a61df37a84440b3f67826ff8c4068" member_type: VOTER last_known_addr { host: "127.8.18.253" port: 39493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 11:33:50.033771 22166 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 837a7216db5341ceb09244a6dd0b6c98: No bootstrap required, opened a new log
W20260810 11:33:50.033671 22364 catalog_manager.cc:1558] T 00000000000000000000000000000000 P af9a61df37a84440b3f67826ff8c4068: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 11:33:50.034107 22364 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 11:33:50.018431 22173 master_runner.cc:186] Error getting master registration for 127.8.18.254:39411: OK, instance_id { permanent_uuid: "837a7216db5341ceb09244a6dd0b6c98" instance_seqno: 1786361628508925 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260810 11:33:50.049036 22173 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260810 11:33:50.051744 22166 raft_consensus.cc:348] T 00000000000000000000000000000000 P 837a7216db5341ceb09244a6dd0b6c98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "837a7216db5341ceb09244a6dd0b6c98" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 39411 } } peers { permanent_uuid: "af9a61df37a84440b3f67826ff8c4068" member_type: VOTER last_known_addr { host: "127.8.18.253" port: 39493 } } peers { permanent_uuid: "e21be99a896b4cce87858590700f425b" member_type: VOTER last_known_addr { host: "127.8.18.252" port: 39373 } }
I20260810 11:33:50.052402 22166 raft_consensus.cc:374] T 00000000000000000000000000000000 P 837a7216db5341ceb09244a6dd0b6c98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:33:50.052618 22166 raft_consensus.cc:729] T 00000000000000000000000000000000 P 837a7216db5341ceb09244a6dd0b6c98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 837a7216db5341ceb09244a6dd0b6c98, State: Initialized, Role: FOLLOWER
I20260810 11:33:50.053673 22166 consensus_queue.cc:260] T 00000000000000000000000000000000 P 837a7216db5341ceb09244a6dd0b6c98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "837a7216db5341ceb09244a6dd0b6c98" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 39411 } } peers { permanent_uuid: "af9a61df37a84440b3f67826ff8c4068" member_type: VOTER last_known_addr { host: "127.8.18.253" port: 39493 } } peers { permanent_uuid: "e21be99a896b4cce87858590700f425b" member_type: VOTER last_known_addr { host: "127.8.18.252" port: 39373 } }
I20260810 11:33:50.061645 22369 sys_catalog.cc:455] T 00000000000000000000000000000000 P 837a7216db5341ceb09244a6dd0b6c98 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "837a7216db5341ceb09244a6dd0b6c98" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 39411 } } peers { permanent_uuid: "af9a61df37a84440b3f67826ff8c4068" member_type: VOTER last_known_addr { host: "127.8.18.253" port: 39493 } } peers { permanent_uuid: "e21be99a896b4cce87858590700f425b" member_type: VOTER last_known_addr { host: "127.8.18.252" port: 39373 } } }
I20260810 11:33:50.063549 22139 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e21be99a896b4cce87858590700f425b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "837a7216db5341ceb09244a6dd0b6c98"
I20260810 11:33:50.070412 22369 sys_catalog.cc:458] T 00000000000000000000000000000000 P 837a7216db5341ceb09244a6dd0b6c98 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 11:33:50.063529 22166 sys_catalog.cc:564] T 00000000000000000000000000000000 P 837a7216db5341ceb09244a6dd0b6c98 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 11:33:50.064128 22138 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 837a7216db5341ceb09244a6dd0b6c98 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:33:50.071363 22348 sys_catalog.cc:455] T 00000000000000000000000000000000 P af9a61df37a84440b3f67826ff8c4068 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e21be99a896b4cce87858590700f425b. Latest consensus state: current_term: 1 leader_uuid: "e21be99a896b4cce87858590700f425b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "837a7216db5341ceb09244a6dd0b6c98" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 39411 } } peers { permanent_uuid: "af9a61df37a84440b3f67826ff8c4068" member_type: VOTER last_known_addr { host: "127.8.18.253" port: 39493 } } peers { permanent_uuid: "e21be99a896b4cce87858590700f425b" member_type: VOTER last_known_addr { host: "127.8.18.252" port: 39373 } } }
I20260810 11:33:50.071731 22348 sys_catalog.cc:458] T 00000000000000000000000000000000 P af9a61df37a84440b3f67826ff8c4068 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 11:33:50.064337 22140 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e21be99a896b4cce87858590700f425b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "837a7216db5341ceb09244a6dd0b6c98" is_pre_election: true
I20260810 11:33:50.077390 22138 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 837a7216db5341ceb09244a6dd0b6c98 [term 1 FOLLOWER]: Refusing update from remote peer e21be99a896b4cce87858590700f425b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 11:33:50.078501 22353 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e21be99a896b4cce87858590700f425b [LEADER]: Connected to new peer: Peer: permanent_uuid: "837a7216db5341ceb09244a6dd0b6c98" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 39411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 11:33:50.081475 22363 mvcc.cc:205] Tried to move back new op lower bound from 7316937236512550912 to 7316937236287807488. Current Snapshot: MvccSnapshot[applied={T|T < 7316937236512550912}]
I20260810 11:33:50.086012 22333 sys_catalog.cc:455] T 00000000000000000000000000000000 P e21be99a896b4cce87858590700f425b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e21be99a896b4cce87858590700f425b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "837a7216db5341ceb09244a6dd0b6c98" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 39411 } } peers { permanent_uuid: "af9a61df37a84440b3f67826ff8c4068" member_type: VOTER last_known_addr { host: "127.8.18.253" port: 39493 } } peers { permanent_uuid: "e21be99a896b4cce87858590700f425b" member_type: VOTER last_known_addr { host: "127.8.18.252" port: 39373 } } }
I20260810 11:33:50.086553 22333 sys_catalog.cc:458] T 00000000000000000000000000000000 P e21be99a896b4cce87858590700f425b [sys.catalog]: This master's current role is: LEADER
I20260810 11:33:50.093794 22360 catalog_manager.cc:1347] Generated new cluster ID: 9b8122e4a85a4036be909884b2526fc9
I20260810 11:33:50.093969 22360 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 11:33:50.110867 22348 sys_catalog.cc:455] T 00000000000000000000000000000000 P af9a61df37a84440b3f67826ff8c4068 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e21be99a896b4cce87858590700f425b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "837a7216db5341ceb09244a6dd0b6c98" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 39411 } } peers { permanent_uuid: "af9a61df37a84440b3f67826ff8c4068" member_type: VOTER last_known_addr { host: "127.8.18.253" port: 39493 } } peers { permanent_uuid: "e21be99a896b4cce87858590700f425b" member_type: VOTER last_known_addr { host: "127.8.18.252" port: 39373 } } }
I20260810 11:33:50.111176 22348 sys_catalog.cc:458] T 00000000000000000000000000000000 P af9a61df37a84440b3f67826ff8c4068 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 11:33:50.114449 22369 sys_catalog.cc:455] T 00000000000000000000000000000000 P 837a7216db5341ceb09244a6dd0b6c98 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e21be99a896b4cce87858590700f425b. Latest consensus state: current_term: 1 leader_uuid: "e21be99a896b4cce87858590700f425b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "837a7216db5341ceb09244a6dd0b6c98" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 39411 } } peers { permanent_uuid: "af9a61df37a84440b3f67826ff8c4068" member_type: VOTER last_known_addr { host: "127.8.18.253" port: 39493 } } peers { permanent_uuid: "e21be99a896b4cce87858590700f425b" member_type: VOTER last_known_addr { host: "127.8.18.252" port: 39373 } } }
I20260810 11:33:50.114802 22369 sys_catalog.cc:458] T 00000000000000000000000000000000 P 837a7216db5341ceb09244a6dd0b6c98 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 11:33:50.116160 22353 sys_catalog.cc:455] T 00000000000000000000000000000000 P e21be99a896b4cce87858590700f425b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e21be99a896b4cce87858590700f425b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "837a7216db5341ceb09244a6dd0b6c98" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 39411 } } peers { permanent_uuid: "af9a61df37a84440b3f67826ff8c4068" member_type: VOTER last_known_addr { host: "127.8.18.253" port: 39493 } } peers { permanent_uuid: "e21be99a896b4cce87858590700f425b" member_type: VOTER last_known_addr { host: "127.8.18.252" port: 39373 } } }
I20260810 11:33:50.116439 22353 sys_catalog.cc:458] T 00000000000000000000000000000000 P e21be99a896b4cce87858590700f425b [sys.catalog]: This master's current role is: LEADER
I20260810 11:33:50.121649 22373 mvcc.cc:205] Tried to move back new op lower bound from 7316937236512550912 to 7316937236287807488. Current Snapshot: MvccSnapshot[applied={T|T < 7316937236512550912}]
I20260810 11:33:50.123445 22369 sys_catalog.cc:455] T 00000000000000000000000000000000 P 837a7216db5341ceb09244a6dd0b6c98 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e21be99a896b4cce87858590700f425b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "837a7216db5341ceb09244a6dd0b6c98" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 39411 } } peers { permanent_uuid: "af9a61df37a84440b3f67826ff8c4068" member_type: VOTER last_known_addr { host: "127.8.18.253" port: 39493 } } peers { permanent_uuid: "e21be99a896b4cce87858590700f425b" member_type: VOTER last_known_addr { host: "127.8.18.252" port: 39373 } } }
I20260810 11:33:50.123792 22369 sys_catalog.cc:458] T 00000000000000000000000000000000 P 837a7216db5341ceb09244a6dd0b6c98 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 11:33:50.141690 22360 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 11:33:50.147320 22360 catalog_manager.cc:1504] Loading token signing keys...
I20260810 11:33:50.167025 22360 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e21be99a896b4cce87858590700f425b: Generated new TSK 0
I20260810 11:33:50.168289 22360 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 11:33:50.287472 22388 catalog_manager.cc:1259] Loaded cluster ID: 9b8122e4a85a4036be909884b2526fc9
I20260810 11:33:50.287703 22388 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 837a7216db5341ceb09244a6dd0b6c98: loading cluster ID for follower catalog manager: success
I20260810 11:33:50.292560 22388 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 837a7216db5341ceb09244a6dd0b6c98: acquiring CA information for follower catalog manager: success
I20260810 11:33:50.296015 22388 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 837a7216db5341ceb09244a6dd0b6c98: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 11:33:51.036453 22364 catalog_manager.cc:1259] Loaded cluster ID: 9b8122e4a85a4036be909884b2526fc9
I20260810 11:33:51.036571 22364 catalog_manager.cc:1552] T 00000000000000000000000000000000 P af9a61df37a84440b3f67826ff8c4068: loading cluster ID for follower catalog manager: success
I20260810 11:33:51.039175 22364 catalog_manager.cc:1574] T 00000000000000000000000000000000 P af9a61df37a84440b3f67826ff8c4068: acquiring CA information for follower catalog manager: success
I20260810 11:33:51.041388 22364 catalog_manager.cc:1602] T 00000000000000000000000000000000 P af9a61df37a84440b3f67826ff8c4068: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 11:33:51.345389 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 22097
I20260810 11:33:51.347787 22161 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:33:51.462687 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 22097
I20260810 11:33:51.489008 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 22173
I20260810 11:33:51.490247 22237 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:33:51.605728 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 22173
I20260810 11:33:51.630091 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 22250
I20260810 11:33:51.631446 22312 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:33:51.751993 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 22250
2026-08-10T11:33:51Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (5251 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786361401922362-8267-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 11:33:51 dist-test-slave-srgv krb5kdc[22397](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 11:33:51 dist-test-slave-srgv krb5kdc[22397](info): set up 2 sockets
Aug 10 11:33:51 dist-test-slave-srgv krb5kdc[22397](info): commencing operation
krb5kdc: starting...
W20260810 11:33:52.969367 8267 mini_kdc.cc:121] Time spent starting KDC: real 1.168s user 0.000s sys 0.039s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786361401922362-8267-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786361401922362-8267-0/krb5kdc/test-user.keytab.
Aug 10 11:33:53 dist-test-slave-srgv krb5kdc[22397](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361633, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T11:33:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T11:33:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.18.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.18.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.18.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.254.keytab.
Entry for principal kudu/127.8.18.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.254.keytab.
I20260810 11:33:53.300271 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786361401922362-8267-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786361401922362-8267-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786361401922362-8267-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.18.254:38141
--webserver_interface=127.8.18.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.254.keytab
--principal=kudu/127.8.18.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.18.212:42483
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.18.254:38141
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786361401922362-8267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786361401922362-8267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786361401922362-8267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:33:53.690593 22410 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:33:53.690994 22410 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:33:53.691155 22410 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:33:53.699235 22410 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 11:33:53.699429 22410 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:33:53.699542 22410 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:33:53.699645 22410 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 11:33:53.699728 22410 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 11:33:53.711014 22410 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.18.212:42483
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786361401922362-8267-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786361401922362-8267-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.18.254:38141
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.254.keytab
--principal=kudu/127.8.18.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.18.254:38141
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.18.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786361401922362-8267-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:33:53.712641 22410 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:33:53.714238 22410 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:33:53.728777 22415 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:33:53.728995 22418 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:33:53.729074 22416 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:33:53.730049 22410 server_base.cc:1034] running on GCE node
I20260810 11:33:53.731034 22410 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:33:53.732856 22410 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:33:53.734171 22410 hybrid_clock.cc:648] HybridClock initialized: now 1786361633734155 us; error 106 us; skew 500 ppm
I20260810 11:33:53.734686 22410 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:33:53 dist-test-slave-srgv krb5kdc[22397](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361633, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:33:53.746259 22410 init.cc:377] Logged in from keytab as kudu/127.8.18.254@KRBTEST.COM (short username kudu)
I20260810 11:33:53.748558 22410 webserver.cc:460] Webserver started at http://127.8.18.254:45255/ using document root <none> and password file <none>
I20260810 11:33:53.749584 22410 fs_manager.cc:359] Metadata directory not provided
I20260810 11:33:53.749775 22410 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:33:53.750196 22410 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:33:53.754016 22410 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786361401922362-8267-0/minicluster-data/master-0/data/instance:
uuid: "246eaac239c4414d91548794785ab862"
format_stamp: "Formatted at 2026-08-10 11:33:53 on dist-test-slave-srgv"
server_key: "1b72307941057e68c69a1b0357c70051"
server_key_iv: "88b01d4533eaa4e7cd8f743296ca1bcc"
server_key_version: "encryptionkey@0"
I20260810 11:33:53.755028 22410 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786361401922362-8267-0/minicluster-data/master-0/wal/instance:
uuid: "246eaac239c4414d91548794785ab862"
format_stamp: "Formatted at 2026-08-10 11:33:53 on dist-test-slave-srgv"
server_key: "1b72307941057e68c69a1b0357c70051"
server_key_iv: "88b01d4533eaa4e7cd8f743296ca1bcc"
server_key_version: "encryptionkey@0"
I20260810 11:33:53.762169 22410 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.003s sys 0.004s
I20260810 11:33:53.766988 22425 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:33:53.768759 22410 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.001s
I20260810 11:33:53.769027 22410 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786361401922362-8267-0/minicluster-data/master-0/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786361401922362-8267-0/minicluster-data/master-0/wal
uuid: "246eaac239c4414d91548794785ab862"
format_stamp: "Formatted at 2026-08-10 11:33:53 on dist-test-slave-srgv"
server_key: "1b72307941057e68c69a1b0357c70051"
server_key_iv: "88b01d4533eaa4e7cd8f743296ca1bcc"
server_key_version: "encryptionkey@0"
I20260810 11:33:53.769315 22410 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786361401922362-8267-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786361401922362-8267-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786361401922362-8267-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:33:53.799559 22410 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:33:53.804673 22410 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:33:53.805024 22410 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:33:53.826912 22410 rpc_server.cc:307] RPC server started. Bound to: 127.8.18.254:38141
I20260810 11:33:53.826936 22478 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.18.254:38141 every 8 connection(s)
I20260810 11:33:53.828660 22410 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
I20260810 11:33:53.833423 22479 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:33:53.834183 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 22410
I20260810 11:33:53.834441 8267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786361401922362-8267-0/minicluster-data/master-0/wal/instance
I20260810 11:33:53.835176 8267 external_mini_cluster.cc:1428] Setting key 31581a536b2f5442ecb031297ded2a7b
I20260810 11:33:53.851456 22479 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 246eaac239c4414d91548794785ab862: Bootstrap starting.
Aug 10 11:33:53 dist-test-slave-srgv krb5kdc[22397](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361633, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.18.254@KRBTEST.COM
I20260810 11:33:53.855252 22479 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 246eaac239c4414d91548794785ab862: Neither blocks nor log segments found. Creating new log.
I20260810 11:33:53.856710 22479 log.cc:824] T 00000000000000000000000000000000 P 246eaac239c4414d91548794785ab862: Log is configured to *not* fsync() on all Append() calls
I20260810 11:33:53.861142 22479 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 246eaac239c4414d91548794785ab862: No bootstrap required, opened a new log
I20260810 11:33:53.866842 22482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:33:53.837522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50212 (local address 127.8.18.254:38141)
0810 11:33:53.838538 (+ 1016us) server_negotiation.cc:206] Beginning negotiation
0810 11:33:53.838562 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0810 11:33:53.838630 (+ 68us) server_negotiation.cc:407] Connection header received
0810 11:33:53.841940 (+ 3310us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:33:53.841985 (+ 45us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:33:53.842742 (+ 757us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:33:53.843365 (+ 623us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:33:53.845200 (+ 1835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:33:53.847533 (+ 2333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:33:53.849284 (+ 1751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:33:53.850555 (+ 1271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:33:53.850815 (+ 260us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:33:53.856242 (+ 5427us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:33:53.856265 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:33:53.856293 (+ 28us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:33:53.856365 (+ 72us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:33:53.860745 (+ 4380us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:33:53.861918 (+ 1173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:33:53.861928 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:33:53.861936 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:33:53.862200 (+ 264us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:33:53.862746 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:33:53.862751 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:33:53.862754 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:33:53.863706 (+ 952us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:33:53.864068 (+ 362us) server_negotiation.cc:1035] Waiting for connection context
0810 11:33:53.864635 (+ 567us) server_negotiation.cc:299] Negotiation successful
0810 11:33:53.865190 (+ 555us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":711,"thread_start_us":403,"threads_started":1}
I20260810 11:33:53.872195 8267 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 11:33:53.872608 22479 raft_consensus.cc:348] T 00000000000000000000000000000000 P 246eaac239c4414d91548794785ab862 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "246eaac239c4414d91548794785ab862" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 38141 } }
I20260810 11:33:53.873091 22479 raft_consensus.cc:374] T 00000000000000000000000000000000 P 246eaac239c4414d91548794785ab862 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:33:53.873255 22479 raft_consensus.cc:729] T 00000000000000000000000000000000 P 246eaac239c4414d91548794785ab862 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 246eaac239c4414d91548794785ab862, State: Initialized, Role: FOLLOWER
I20260810 11:33:53.873997 22479 consensus_queue.cc:260] T 00000000000000000000000000000000 P 246eaac239c4414d91548794785ab862 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "246eaac239c4414d91548794785ab862" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 38141 } }
I20260810 11:33:53.874383 22479 raft_consensus.cc:388] T 00000000000000000000000000000000 P 246eaac239c4414d91548794785ab862 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 11:33:53.874562 22479 raft_consensus.cc:482] T 00000000000000000000000000000000 P 246eaac239c4414d91548794785ab862 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 11:33:53.874788 22479 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 246eaac239c4414d91548794785ab862 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:33:53.878114 22479 raft_consensus.cc:504] T 00000000000000000000000000000000 P 246eaac239c4414d91548794785ab862 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "246eaac239c4414d91548794785ab862" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 38141 } }
I20260810 11:33:53.878777 22479 leader_election.cc:302] T 00000000000000000000000000000000 P 246eaac239c4414d91548794785ab862 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 246eaac239c4414d91548794785ab862; no voters:
I20260810 11:33:53.879482 22479 leader_election.cc:288] T 00000000000000000000000000000000 P 246eaac239c4414d91548794785ab862 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 11:33:53.879671 22485 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 246eaac239c4414d91548794785ab862 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:33:53.880512 22485 raft_consensus.cc:686] T 00000000000000000000000000000000 P 246eaac239c4414d91548794785ab862 [term 1 LEADER]: Becoming Leader. State: Replica: 246eaac239c4414d91548794785ab862, State: Running, Role: LEADER
I20260810 11:33:53.881178 22485 consensus_queue.cc:237] T 00000000000000000000000000000000 P 246eaac239c4414d91548794785ab862 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "246eaac239c4414d91548794785ab862" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 38141 } }
I20260810 11:33:53.882072 22479 sys_catalog.cc:564] T 00000000000000000000000000000000 P 246eaac239c4414d91548794785ab862 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 11:33:53.884642 22486 sys_catalog.cc:455] T 00000000000000000000000000000000 P 246eaac239c4414d91548794785ab862 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "246eaac239c4414d91548794785ab862" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "246eaac239c4414d91548794785ab862" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 38141 } } }
I20260810 11:33:53.885228 22486 sys_catalog.cc:458] T 00000000000000000000000000000000 P 246eaac239c4414d91548794785ab862 [sys.catalog]: This master's current role is: LEADER
I20260810 11:33:53.884735 22487 sys_catalog.cc:455] T 00000000000000000000000000000000 P 246eaac239c4414d91548794785ab862 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 246eaac239c4414d91548794785ab862. Latest consensus state: current_term: 1 leader_uuid: "246eaac239c4414d91548794785ab862" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "246eaac239c4414d91548794785ab862" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 38141 } } }
I20260810 11:33:53.885563 22487 sys_catalog.cc:458] T 00000000000000000000000000000000 P 246eaac239c4414d91548794785ab862 [sys.catalog]: This master's current role is: LEADER
I20260810 11:33:53.886343 22493 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 11:33:53.891912 22493 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 11:33:53.903245 22493 catalog_manager.cc:1347] Generated new cluster ID: 4575bafeab69459db9a397599cf70639
I20260810 11:33:53.903424 22493 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 11:33:53.914126 22493 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 11:33:53.915679 22493 catalog_manager.cc:1504] Loading token signing keys...
I20260810 11:33:53.927459 22493 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 246eaac239c4414d91548794785ab862: Generated new TSK 0
I20260810 11:33:53.928773 22493 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 11:33:53.987882 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 22410
I20260810 11:33:53.989696 22474 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:33:54.095820 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 22410
2026-08-10T11:33:54Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (2336 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 11:33:54 dist-test-slave-srgv krb5kdc[22509](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 11:33:54 dist-test-slave-srgv krb5kdc[22509](info): set up 2 sockets
Aug 10 11:33:54 dist-test-slave-srgv krb5kdc[22509](info): commencing operation
krb5kdc: starting...
W20260810 11:33:55.275420 8267 mini_kdc.cc:121] Time spent starting KDC: real 1.138s user 0.000s sys 0.037s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/krb5kdc/test-user.keytab.
Aug 10 11:33:55 dist-test-slave-srgv krb5kdc[22509](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361635, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T11:33:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T11:33:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.18.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.18.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.18.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.254.keytab.
Entry for principal kudu/127.8.18.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.254.keytab.
I20260810 11:33:55.615550 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.18.254:43799
--webserver_interface=127.8.18.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.254.keytab
--principal=kudu/127.8.18.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.18.212:35253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.18.254:43799
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:33:56.014647 22522 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:33:56.015007 22522 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:33:56.015225 22522 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:33:56.023316 22522 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 11:33:56.023469 22522 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:33:56.023563 22522 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:33:56.023677 22522 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 11:33:56.023759 22522 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 11:33:56.035379 22522 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.18.212:35253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.18.254:43799
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.254.keytab
--principal=kudu/127.8.18.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.18.254:43799
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.18.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:33:56.037102 22522 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:33:56.038631 22522 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:33:56.052381 22527 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:33:56.052456 22530 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:33:56.052425 22528 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:33:56.053596 22522 server_base.cc:1034] running on GCE node
I20260810 11:33:56.054546 22522 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:33:56.056177 22522 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:33:56.057540 22522 hybrid_clock.cc:648] HybridClock initialized: now 1786361636057486 us; error 43 us; skew 500 ppm
I20260810 11:33:56.057994 22522 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:33:56 dist-test-slave-srgv krb5kdc[22509](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361636, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:33:56.061856 22522 init.cc:377] Logged in from keytab as kudu/127.8.18.254@KRBTEST.COM (short username kudu)
I20260810 11:33:56.063869 22522 webserver.cc:460] Webserver started at http://127.8.18.254:40949/ using document root <none> and password file <none>
I20260810 11:33:56.064759 22522 fs_manager.cc:359] Metadata directory not provided
I20260810 11:33:56.064965 22522 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:33:56.065332 22522 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:33:56.068905 22522 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/master-0/data/instance:
uuid: "99954a220cf4425b894a741dd117548a"
format_stamp: "Formatted at 2026-08-10 11:33:56 on dist-test-slave-srgv"
server_key: "dd0a6aa35d36a6fdd867b117088c8f9a"
server_key_iv: "6239f1ab4e6711ccf45bd557db90dd00"
server_key_version: "encryptionkey@0"
I20260810 11:33:56.070546 22522 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/master-0/wal/instance:
uuid: "99954a220cf4425b894a741dd117548a"
format_stamp: "Formatted at 2026-08-10 11:33:56 on dist-test-slave-srgv"
server_key: "dd0a6aa35d36a6fdd867b117088c8f9a"
server_key_iv: "6239f1ab4e6711ccf45bd557db90dd00"
server_key_version: "encryptionkey@0"
I20260810 11:33:56.077536 22522 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260810 11:33:56.082017 22538 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:33:56.083779 22522 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.004s
I20260810 11:33:56.084079 22522 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/master-0/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/master-0/wal
uuid: "99954a220cf4425b894a741dd117548a"
format_stamp: "Formatted at 2026-08-10 11:33:56 on dist-test-slave-srgv"
server_key: "dd0a6aa35d36a6fdd867b117088c8f9a"
server_key_iv: "6239f1ab4e6711ccf45bd557db90dd00"
server_key_version: "encryptionkey@0"
I20260810 11:33:56.084393 22522 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:33:56.100407 22522 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:33:56.105839 22522 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:33:56.106189 22522 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:33:56.128742 22522 rpc_server.cc:307] RPC server started. Bound to: 127.8.18.254:43799
I20260810 11:33:56.128767 22590 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.18.254:43799 every 8 connection(s)
I20260810 11:33:56.130563 22522 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
I20260810 11:33:56.135130 22591 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:33:56.139909 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 22522
I20260810 11:33:56.140194 8267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/master-0/wal/instance
I20260810 11:33:56.141074 8267 external_mini_cluster.cc:1428] Setting key f7204089771c8cd7f24d9b3d22a6a5b0
I20260810 11:33:56.149148 22591 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 99954a220cf4425b894a741dd117548a: Bootstrap starting.
I20260810 11:33:56.153467 22591 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 99954a220cf4425b894a741dd117548a: Neither blocks nor log segments found. Creating new log.
I20260810 11:33:56.154922 22591 log.cc:824] T 00000000000000000000000000000000 P 99954a220cf4425b894a741dd117548a: Log is configured to *not* fsync() on all Append() calls
I20260810 11:33:56.159577 22591 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 99954a220cf4425b894a741dd117548a: No bootstrap required, opened a new log
Aug 10 11:33:56 dist-test-slave-srgv krb5kdc[22509](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361635, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.18.254@KRBTEST.COM
I20260810 11:33:56.171567 22591 raft_consensus.cc:348] T 00000000000000000000000000000000 P 99954a220cf4425b894a741dd117548a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99954a220cf4425b894a741dd117548a" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 43799 } }
I20260810 11:33:56.172084 22591 raft_consensus.cc:374] T 00000000000000000000000000000000 P 99954a220cf4425b894a741dd117548a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:33:56.172231 22591 raft_consensus.cc:729] T 00000000000000000000000000000000 P 99954a220cf4425b894a741dd117548a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99954a220cf4425b894a741dd117548a, State: Initialized, Role: FOLLOWER
I20260810 11:33:56.172960 22591 consensus_queue.cc:260] T 00000000000000000000000000000000 P 99954a220cf4425b894a741dd117548a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99954a220cf4425b894a741dd117548a" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 43799 } }
I20260810 11:33:56.173272 22591 raft_consensus.cc:388] T 00000000000000000000000000000000 P 99954a220cf4425b894a741dd117548a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 11:33:56.173362 22594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:33:56.143665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40582 (local address 127.8.18.254:43799)
0810 11:33:56.144671 (+ 1006us) server_negotiation.cc:206] Beginning negotiation
0810 11:33:56.144685 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 11:33:56.144739 (+ 54us) server_negotiation.cc:407] Connection header received
0810 11:33:56.147426 (+ 2687us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:33:56.147462 (+ 36us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:33:56.147978 (+ 516us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:33:56.148396 (+ 418us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:33:56.150600 (+ 2204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:33:56.152146 (+ 1546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:33:56.154332 (+ 2186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:33:56.155285 (+ 953us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:33:56.155461 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:33:56.162106 (+ 6645us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:33:56.162121 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:33:56.162158 (+ 37us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:33:56.162220 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:33:56.167123 (+ 4903us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:33:56.168271 (+ 1148us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:33:56.168281 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:33:56.168292 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:33:56.168560 (+ 268us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:33:56.169077 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:33:56.169083 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:33:56.169087 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:33:56.169968 (+ 881us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:33:56.170266 (+ 298us) server_negotiation.cc:1035] Waiting for connection context
0810 11:33:56.171270 (+ 1004us) server_negotiation.cc:299] Negotiation successful
0810 11:33:56.171972 (+ 702us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":749,"thread_start_us":467,"threads_started":1}
I20260810 11:33:56.173473 22591 raft_consensus.cc:482] T 00000000000000000000000000000000 P 99954a220cf4425b894a741dd117548a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 11:33:56.173719 22591 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 99954a220cf4425b894a741dd117548a [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:33:56.177024 22591 raft_consensus.cc:504] T 00000000000000000000000000000000 P 99954a220cf4425b894a741dd117548a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99954a220cf4425b894a741dd117548a" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 43799 } }
I20260810 11:33:56.177702 22591 leader_election.cc:302] T 00000000000000000000000000000000 P 99954a220cf4425b894a741dd117548a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 99954a220cf4425b894a741dd117548a; no voters:
I20260810 11:33:56.178354 22591 leader_election.cc:288] T 00000000000000000000000000000000 P 99954a220cf4425b894a741dd117548a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 11:33:56.178514 22596 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 99954a220cf4425b894a741dd117548a [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:33:56.179661 22596 raft_consensus.cc:686] T 00000000000000000000000000000000 P 99954a220cf4425b894a741dd117548a [term 1 LEADER]: Becoming Leader. State: Replica: 99954a220cf4425b894a741dd117548a, State: Running, Role: LEADER
I20260810 11:33:56.180264 22596 consensus_queue.cc:237] T 00000000000000000000000000000000 P 99954a220cf4425b894a741dd117548a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99954a220cf4425b894a741dd117548a" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 43799 } }
I20260810 11:33:56.180593 22591 sys_catalog.cc:564] T 00000000000000000000000000000000 P 99954a220cf4425b894a741dd117548a [sys.catalog]: configured and running, proceeding with master startup.
I20260810 11:33:56.183740 22597 sys_catalog.cc:455] T 00000000000000000000000000000000 P 99954a220cf4425b894a741dd117548a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "99954a220cf4425b894a741dd117548a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99954a220cf4425b894a741dd117548a" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 43799 } } }
I20260810 11:33:56.184309 22597 sys_catalog.cc:458] T 00000000000000000000000000000000 P 99954a220cf4425b894a741dd117548a [sys.catalog]: This master's current role is: LEADER
I20260810 11:33:56.185107 22598 sys_catalog.cc:455] T 00000000000000000000000000000000 P 99954a220cf4425b894a741dd117548a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 99954a220cf4425b894a741dd117548a. Latest consensus state: current_term: 1 leader_uuid: "99954a220cf4425b894a741dd117548a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99954a220cf4425b894a741dd117548a" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 43799 } } }
I20260810 11:33:56.185593 22598 sys_catalog.cc:458] T 00000000000000000000000000000000 P 99954a220cf4425b894a741dd117548a [sys.catalog]: This master's current role is: LEADER
I20260810 11:33:56.187259 22604 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 11:33:56.193608 22604 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 11:33:56.205443 22604 catalog_manager.cc:1347] Generated new cluster ID: b7dd071c324847619421b3caaec5f6e3
I20260810 11:33:56.205598 22604 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 11:33:56.248093 22604 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 11:33:56.249887 22604 catalog_manager.cc:1504] Loading token signing keys...
I20260810 11:33:56.261765 22604 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 99954a220cf4425b894a741dd117548a: Generated new TSK 0
I20260810 11:33:56.263124 22604 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.18.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.18.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.18.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.193.keytab.
Entry for principal kudu/127.8.18.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.193.keytab.
I20260810 11:33:56.355175 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.18.193:0
--local_ip_for_outbound_sockets=127.8.18.193
--webserver_interface=127.8.18.193
--webserver_port=0
--tserver_master_addrs=127.8.18.254:43799
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.193.keytab
--principal=kudu/127.8.18.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.18.212:35253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:33:56.749780 22617 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:33:56.750118 22617 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:33:56.750306 22617 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:33:56.758242 22617 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:33:56.758396 22617 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:33:56.758591 22617 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.18.193
I20260810 11:33:56.769850 22617 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.18.212:35253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.193.keytab
--principal=kudu/127.8.18.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.18.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.18.193
--webserver_port=0
--tserver_master_addrs=127.8.18.254:43799
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.18.193
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:33:56.771356 22617 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:33:56.772878 22617 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:33:56.789108 22622 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:33:56.789183 22623 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:33:56.789458 22625 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:33:56.790721 22617 server_base.cc:1034] running on GCE node
I20260810 11:33:56.791651 22617 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:33:56.792866 22617 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:33:56.794178 22617 hybrid_clock.cc:648] HybridClock initialized: now 1786361636794082 us; error 46 us; skew 500 ppm
I20260810 11:33:56.794555 22617 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:33:56 dist-test-slave-srgv krb5kdc[22509](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361636, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:33:56.798373 22617 init.cc:377] Logged in from keytab as kudu/127.8.18.193@KRBTEST.COM (short username kudu)
I20260810 11:33:56.800283 22617 webserver.cc:460] Webserver started at http://127.8.18.193:43503/ using document root <none> and password file <none>
I20260810 11:33:56.801164 22617 fs_manager.cc:359] Metadata directory not provided
I20260810 11:33:56.801339 22617 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:33:56.801784 22617 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:33:56.805317 22617 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/ts-0/data/instance:
uuid: "b53ea98f7b4b4a828f90fc4578efe84a"
format_stamp: "Formatted at 2026-08-10 11:33:56 on dist-test-slave-srgv"
server_key: "432487eb597c21bb93993d7fff75c65d"
server_key_iv: "fd74c204c2a17562810d2d473e2f02f4"
server_key_version: "encryptionkey@0"
I20260810 11:33:56.806412 22617 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/ts-0/wal/instance:
uuid: "b53ea98f7b4b4a828f90fc4578efe84a"
format_stamp: "Formatted at 2026-08-10 11:33:56 on dist-test-slave-srgv"
server_key: "432487eb597c21bb93993d7fff75c65d"
server_key_iv: "fd74c204c2a17562810d2d473e2f02f4"
server_key_version: "encryptionkey@0"
I20260810 11:33:56.812865 22617 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260810 11:33:56.817696 22633 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:33:56.819499 22617 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260810 11:33:56.819732 22617 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/ts-0/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/ts-0/wal
uuid: "b53ea98f7b4b4a828f90fc4578efe84a"
format_stamp: "Formatted at 2026-08-10 11:33:56 on dist-test-slave-srgv"
server_key: "432487eb597c21bb93993d7fff75c65d"
server_key_iv: "fd74c204c2a17562810d2d473e2f02f4"
server_key_version: "encryptionkey@0"
I20260810 11:33:56.819988 22617 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:33:56.847419 22617 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:33:56.852967 22617 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:33:56.853358 22617 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:33:56.854611 22617 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:33:56.856191 22617 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:33:56.856276 22617 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:33:56.856449 22617 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:33:56.856523 22617 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:33:56.896010 22617 rpc_server.cc:307] RPC server started. Bound to: 127.8.18.193:41937
I20260810 11:33:56.896060 22746 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.18.193:41937 every 8 connection(s)
I20260810 11:33:56.897505 22617 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/ts-0/data/info.pb
I20260810 11:33:56.900614 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 22617
I20260810 11:33:56.900870 8267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/ts-0/wal/instance
I20260810 11:33:56.901786 8267 external_mini_cluster.cc:1428] Setting key 690eadc173560b91b9b31755d55fec77
Aug 10 11:33:56 dist-test-slave-srgv krb5kdc[22509](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361636, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.193@KRBTEST.COM for kudu/127.8.18.254@KRBTEST.COM
I20260810 11:33:56.920562 22749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:33:56.900556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.193:33445 (local address 127.8.18.254:43799)
0810 11:33:56.901065 (+ 509us) server_negotiation.cc:206] Beginning negotiation
0810 11:33:56.901069 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 11:33:56.902465 (+ 1396us) server_negotiation.cc:407] Connection header received
0810 11:33:56.904906 (+ 2441us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:33:56.904912 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:33:56.905020 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:33:56.905147 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:33:56.906951 (+ 1804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:33:56.907698 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:33:56.909456 (+ 1758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:33:56.909880 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:33:56.910038 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:33:56.914097 (+ 4059us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:33:56.914103 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:33:56.914106 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:33:56.914147 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:33:56.917040 (+ 2893us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:33:56.918003 (+ 963us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:33:56.918008 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:33:56.918012 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:33:56.918157 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:33:56.918654 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:33:56.918659 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:33:56.918663 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:33:56.918977 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:33:56.919143 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0810 11:33:56.919924 (+ 781us) server_negotiation.cc:299] Negotiation successful
0810 11:33:56.920183 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":405,"thread_start_us":294,"threads_started":1}
I20260810 11:33:56.921797 22750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:33:56.901260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:43799 (local address 127.8.18.193:33445)
0810 11:33:56.902276 (+ 1016us) negotiation.cc:107] Waiting for socket to connect
0810 11:33:56.902328 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0810 11:33:56.904593 (+ 2265us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:33:56.905482 (+ 889us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:33:56.905493 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:33:56.906065 (+ 572us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:33:56.906749 (+ 684us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:33:56.906770 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:33:56.907888 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:33:56.907892 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:33:56.909244 (+ 1352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:33:56.909257 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:33:56.910163 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:33:56.910172 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:33:56.910433 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:33:56.911234 (+ 801us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:33:56.911260 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:33:56.913793 (+ 2533us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:33:56.917234 (+ 3441us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:33:56.917244 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:33:56.917272 (+ 28us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:33:56.917855 (+ 583us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:33:56.918284 (+ 429us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:33:56.918290 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:33:56.918293 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:33:56.918511 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:33:56.919128 (+ 617us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:33:56.919136 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:33:56.919514 (+ 378us) client_negotiation.cc:769] Sending connection context
0810 11:33:56.919807 (+ 293us) client_negotiation.cc:240] Negotiation successful
0810 11:33:56.920429 (+ 622us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":710,"thread_start_us":390,"threads_started":1}
I20260810 11:33:56.923794 22747 heartbeater.cc:347] Connected to a master server at 127.8.18.254:43799
I20260810 11:33:56.924209 22747 heartbeater.cc:464] Registering TS with master...
I20260810 11:33:56.925221 22747 heartbeater.cc:511] Master 127.8.18.254:43799 requested a full tablet report, sending...
I20260810 11:33:56.928124 22555 ts_manager.cc:194] Registered new tserver with Master: b53ea98f7b4b4a828f90fc4578efe84a (127.8.18.193:41937)
I20260810 11:33:56.930416 22555 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.18.193@KRBTEST.COM'} at 127.8.18.193:33445
WARNING: no policy specified for kudu/127.8.18.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.18.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.18.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.194.keytab.
Entry for principal kudu/127.8.18.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.194.keytab.
I20260810 11:33:56.969532 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.18.194:0
--local_ip_for_outbound_sockets=127.8.18.194
--webserver_interface=127.8.18.194
--webserver_port=0
--tserver_master_addrs=127.8.18.254:43799
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.194.keytab
--principal=kudu/127.8.18.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.18.212:35253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:33:57.349967 22753 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:33:57.350282 22753 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:33:57.350435 22753 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:33:57.358533 22753 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:33:57.358678 22753 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:33:57.358875 22753 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.18.194
I20260810 11:33:57.369572 22753 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.18.212:35253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.194.keytab
--principal=kudu/127.8.18.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.18.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.18.194
--webserver_port=0
--tserver_master_addrs=127.8.18.254:43799
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.18.194
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:33:57.371034 22753 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:33:57.372491 22753 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:33:57.387447 22758 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:33:57.388017 22760 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:33:57.388288 22762 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:33:57.388753 22753 server_base.cc:1034] running on GCE node
I20260810 11:33:57.389597 22753 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:33:57.390733 22753 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:33:57.392010 22753 hybrid_clock.cc:648] HybridClock initialized: now 1786361637391932 us; error 91 us; skew 500 ppm
I20260810 11:33:57.392477 22753 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:33:57 dist-test-slave-srgv krb5kdc[22509](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361637, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:33:57.396271 22753 init.cc:377] Logged in from keytab as kudu/127.8.18.194@KRBTEST.COM (short username kudu)
I20260810 11:33:57.398135 22753 webserver.cc:460] Webserver started at http://127.8.18.194:36647/ using document root <none> and password file <none>
I20260810 11:33:57.398969 22753 fs_manager.cc:359] Metadata directory not provided
I20260810 11:33:57.399134 22753 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:33:57.399555 22753 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:33:57.403591 22753 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/ts-1/data/instance:
uuid: "abbaeefbabb941d8ae31fe6e637fdbc0"
format_stamp: "Formatted at 2026-08-10 11:33:57 on dist-test-slave-srgv"
server_key: "42d45e663bfadeb03076cdff4ecf7fef"
server_key_iv: "c89e0cbaa3e21eb63f117d46dd0343af"
server_key_version: "encryptionkey@0"
I20260810 11:33:57.404681 22753 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/ts-1/wal/instance:
uuid: "abbaeefbabb941d8ae31fe6e637fdbc0"
format_stamp: "Formatted at 2026-08-10 11:33:57 on dist-test-slave-srgv"
server_key: "42d45e663bfadeb03076cdff4ecf7fef"
server_key_iv: "c89e0cbaa3e21eb63f117d46dd0343af"
server_key_version: "encryptionkey@0"
I20260810 11:33:57.411486 22753 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.009s sys 0.000s
I20260810 11:33:57.416350 22769 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:33:57.418181 22753 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260810 11:33:57.418438 22753 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/ts-1/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/ts-1/wal
uuid: "abbaeefbabb941d8ae31fe6e637fdbc0"
format_stamp: "Formatted at 2026-08-10 11:33:57 on dist-test-slave-srgv"
server_key: "42d45e663bfadeb03076cdff4ecf7fef"
server_key_iv: "c89e0cbaa3e21eb63f117d46dd0343af"
server_key_version: "encryptionkey@0"
I20260810 11:33:57.418701 22753 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:33:57.454845 22753 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:33:57.460223 22753 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:33:57.460613 22753 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:33:57.461814 22753 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:33:57.463425 22753 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:33:57.463546 22753 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:33:57.463726 22753 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:33:57.463804 22753 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:33:57.502302 22753 rpc_server.cc:307] RPC server started. Bound to: 127.8.18.194:44985
I20260810 11:33:57.502368 22882 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.18.194:44985 every 8 connection(s)
I20260810 11:33:57.503852 22753 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/ts-1/data/info.pb
I20260810 11:33:57.512111 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 22753
I20260810 11:33:57.512327 8267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/ts-1/wal/instance
I20260810 11:33:57.512822 8267 external_mini_cluster.cc:1428] Setting key 68fe744c11d0f49a1a5ce7d564e555c5
Aug 10 11:33:57 dist-test-slave-srgv krb5kdc[22509](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361637, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.194@KRBTEST.COM for kudu/127.8.18.254@KRBTEST.COM
I20260810 11:33:57.529300 22885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:33:57.506922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.194:48863 (local address 127.8.18.254:43799)
0810 11:33:57.507406 (+ 484us) server_negotiation.cc:206] Beginning negotiation
0810 11:33:57.507410 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 11:33:57.508538 (+ 1128us) server_negotiation.cc:407] Connection header received
0810 11:33:57.510727 (+ 2189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:33:57.510733 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:33:57.510850 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:33:57.510978 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:33:57.512830 (+ 1852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:33:57.513839 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:33:57.515911 (+ 2072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:33:57.516361 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:33:57.516480 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:33:57.521536 (+ 5056us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:33:57.521542 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:33:57.521546 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:33:57.521588 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:33:57.525340 (+ 3752us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:33:57.526454 (+ 1114us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:33:57.526463 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:33:57.526467 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:33:57.526631 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:33:57.527181 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:33:57.527187 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:33:57.527191 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:33:57.527533 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:33:57.527730 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0810 11:33:57.528555 (+ 825us) server_negotiation.cc:299] Negotiation successful
0810 11:33:57.528870 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":371,"thread_start_us":244,"threads_started":1}
I20260810 11:33:57.530764 22886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:33:57.507565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:43799 (local address 127.8.18.194:48863)
0810 11:33:57.508328 (+ 763us) negotiation.cc:107] Waiting for socket to connect
0810 11:33:57.508372 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 11:33:57.510374 (+ 2002us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:33:57.511273 (+ 899us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:33:57.511282 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:33:57.511866 (+ 584us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:33:57.512619 (+ 753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:33:57.512642 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:33:57.514025 (+ 1383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:33:57.514030 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:33:57.515692 (+ 1662us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:33:57.515713 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:33:57.516656 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:33:57.516666 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:33:57.517062 (+ 396us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:33:57.518003 (+ 941us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:33:57.518047 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:33:57.521163 (+ 3116us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:33:57.525635 (+ 4472us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:33:57.525649 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:33:57.525671 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:33:57.526271 (+ 600us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:33:57.526786 (+ 515us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:33:57.526792 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:33:57.526796 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:33:57.527038 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:33:57.527693 (+ 655us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:33:57.527702 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:33:57.528125 (+ 423us) client_negotiation.cc:769] Sending connection context
0810 11:33:57.528461 (+ 336us) client_negotiation.cc:240] Negotiation successful
0810 11:33:57.529112 (+ 651us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":507,"thread_start_us":306,"threads_started":1}
I20260810 11:33:57.532546 22883 heartbeater.cc:347] Connected to a master server at 127.8.18.254:43799
I20260810 11:33:57.532941 22883 heartbeater.cc:464] Registering TS with master...
I20260810 11:33:57.534076 22883 heartbeater.cc:511] Master 127.8.18.254:43799 requested a full tablet report, sending...
I20260810 11:33:57.536077 22555 ts_manager.cc:194] Registered new tserver with Master: abbaeefbabb941d8ae31fe6e637fdbc0 (127.8.18.194:44985)
I20260810 11:33:57.537516 22555 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.18.194@KRBTEST.COM'} at 127.8.18.194:48863
WARNING: no policy specified for kudu/127.8.18.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.18.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.18.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.195.keytab.
Entry for principal kudu/127.8.18.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.195.keytab.
I20260810 11:33:57.581539 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.18.195:0
--local_ip_for_outbound_sockets=127.8.18.195
--webserver_interface=127.8.18.195
--webserver_port=0
--tserver_master_addrs=127.8.18.254:43799
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.195.keytab
--principal=kudu/127.8.18.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.18.212:35253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 11:33:57.934034 22747 heartbeater.cc:503] Master 127.8.18.254:43799 was elected leader, sending a full tablet report...
W20260810 11:33:57.978245 22889 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:33:57.978581 22889 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:33:57.978740 22889 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:33:57.986780 22889 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:33:57.986919 22889 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:33:57.987140 22889 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.18.195
I20260810 11:33:57.998088 22889 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.18.212:35253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.195.keytab
--principal=kudu/127.8.18.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.18.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.18.195
--webserver_port=0
--tserver_master_addrs=127.8.18.254:43799
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.18.195
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:33:57.999593 22889 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:33:58.001106 22889 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:33:58.016583 22894 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:33:58.016561 22896 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:33:58.016525 22898 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:33:58.017446 22889 server_base.cc:1034] running on GCE node
I20260810 11:33:58.018455 22889 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:33:58.019883 22889 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:33:58.021207 22889 hybrid_clock.cc:648] HybridClock initialized: now 1786361638021147 us; error 46 us; skew 500 ppm
I20260810 11:33:58.021723 22889 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:33:58 dist-test-slave-srgv krb5kdc[22509](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361638, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:33:58.026173 22889 init.cc:377] Logged in from keytab as kudu/127.8.18.195@KRBTEST.COM (short username kudu)
I20260810 11:33:58.027922 22889 webserver.cc:460] Webserver started at http://127.8.18.195:35623/ using document root <none> and password file <none>
I20260810 11:33:58.028746 22889 fs_manager.cc:359] Metadata directory not provided
I20260810 11:33:58.028894 22889 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:33:58.029248 22889 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:33:58.032608 22889 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/ts-2/data/instance:
uuid: "52042505dd7f42e990ee5684ada73014"
format_stamp: "Formatted at 2026-08-10 11:33:58 on dist-test-slave-srgv"
server_key: "5de592d87e7c1bb273f31396872e2cd2"
server_key_iv: "50211b8d617be8277b70ee6472c7f469"
server_key_version: "encryptionkey@0"
I20260810 11:33:58.033594 22889 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/ts-2/wal/instance:
uuid: "52042505dd7f42e990ee5684ada73014"
format_stamp: "Formatted at 2026-08-10 11:33:58 on dist-test-slave-srgv"
server_key: "5de592d87e7c1bb273f31396872e2cd2"
server_key_iv: "50211b8d617be8277b70ee6472c7f469"
server_key_version: "encryptionkey@0"
I20260810 11:33:58.039682 22889 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260810 11:33:58.043998 22905 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:33:58.045758 22889 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260810 11:33:58.045967 22889 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/ts-2/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/ts-2/wal
uuid: "52042505dd7f42e990ee5684ada73014"
format_stamp: "Formatted at 2026-08-10 11:33:58 on dist-test-slave-srgv"
server_key: "5de592d87e7c1bb273f31396872e2cd2"
server_key_iv: "50211b8d617be8277b70ee6472c7f469"
server_key_version: "encryptionkey@0"
I20260810 11:33:58.046203 22889 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:33:58.069347 22889 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:33:58.074420 22889 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:33:58.074771 22889 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:33:58.075882 22889 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:33:58.077495 22889 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:33:58.077610 22889 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:33:58.077773 22889 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:33:58.077850 22889 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:33:58.115096 22889 rpc_server.cc:307] RPC server started. Bound to: 127.8.18.195:43839
I20260810 11:33:58.115192 23018 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.18.195:43839 every 8 connection(s)
I20260810 11:33:58.116523 22889 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/ts-2/data/info.pb
I20260810 11:33:58.124951 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 22889
I20260810 11:33:58.125231 8267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/ts-2/wal/instance
I20260810 11:33:58.125838 8267 external_mini_cluster.cc:1428] Setting key 77cfb8f25456319859d939bcad0406f8
Aug 10 11:33:58 dist-test-slave-srgv krb5kdc[22509](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361638, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.195@KRBTEST.COM for kudu/127.8.18.254@KRBTEST.COM
I20260810 11:33:58.141757 23021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:33:58.119465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.195:34911 (local address 127.8.18.254:43799)
0810 11:33:58.119977 (+ 512us) server_negotiation.cc:206] Beginning negotiation
0810 11:33:58.119983 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:33:58.120835 (+ 852us) server_negotiation.cc:407] Connection header received
0810 11:33:58.122965 (+ 2130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:33:58.122972 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:33:58.123080 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:33:58.123206 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:33:58.125109 (+ 1903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:33:58.126129 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:33:58.128037 (+ 1908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:33:58.128566 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:33:58.128708 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:33:58.133308 (+ 4600us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:33:58.133316 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:33:58.133322 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:33:58.133371 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:33:58.137608 (+ 4237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:33:58.138587 (+ 979us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:33:58.138594 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:33:58.138598 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:33:58.138784 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:33:58.139280 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:33:58.139285 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:33:58.139290 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:33:58.139699 (+ 409us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:33:58.139880 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0810 11:33:58.140931 (+ 1051us) server_negotiation.cc:299] Negotiation successful
0810 11:33:58.141282 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":413,"thread_start_us":274,"threads_started":1}
I20260810 11:33:58.143000 23022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:33:58.119887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:43799 (local address 127.8.18.195:34911)
0810 11:33:58.120657 (+ 770us) negotiation.cc:107] Waiting for socket to connect
0810 11:33:58.120697 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0810 11:33:58.122625 (+ 1928us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:33:58.123502 (+ 877us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:33:58.123515 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:33:58.124200 (+ 685us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:33:58.124886 (+ 686us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:33:58.124924 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:33:58.126318 (+ 1394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:33:58.126322 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:33:58.127862 (+ 1540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:33:58.127876 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:33:58.128819 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:33:58.128826 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:33:58.129084 (+ 258us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:33:58.130001 (+ 917us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:33:58.130046 (+ 45us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:33:58.133021 (+ 2975us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:33:58.137838 (+ 4817us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:33:58.137847 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:33:58.137864 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:33:58.138411 (+ 547us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:33:58.138925 (+ 514us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:33:58.138929 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:33:58.138931 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:33:58.139159 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:33:58.139843 (+ 684us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:33:58.139850 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:33:58.140406 (+ 556us) client_negotiation.cc:769] Sending connection context
0810 11:33:58.140776 (+ 370us) client_negotiation.cc:240] Negotiation successful
0810 11:33:58.141455 (+ 679us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":530,"thread_start_us":313,"threads_started":1}
I20260810 11:33:58.144560 23019 heartbeater.cc:347] Connected to a master server at 127.8.18.254:43799
I20260810 11:33:58.144927 23019 heartbeater.cc:464] Registering TS with master...
I20260810 11:33:58.145816 23019 heartbeater.cc:511] Master 127.8.18.254:43799 requested a full tablet report, sending...
I20260810 11:33:58.147581 22555 ts_manager.cc:194] Registered new tserver with Master: 52042505dd7f42e990ee5684ada73014 (127.8.18.195:43839)
I20260810 11:33:58.148497 22555 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.18.195@KRBTEST.COM'} at 127.8.18.195:34911
I20260810 11:33:58.156241 8267 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 11:33:58.175753 23021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:33:58.161531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48892 (local address 127.8.18.254:43799)
0810 11:33:58.161805 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0810 11:33:58.161812 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:33:58.162117 (+ 305us) server_negotiation.cc:407] Connection header received
0810 11:33:58.162282 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:33:58.162286 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:33:58.162386 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:33:58.162476 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:33:58.164013 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:33:58.164772 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:33:58.166407 (+ 1635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:33:58.166842 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:33:58.166942 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:33:58.169757 (+ 2815us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:33:58.169762 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:33:58.169766 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:33:58.169805 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:33:58.172498 (+ 2693us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:33:58.173469 (+ 971us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:33:58.173474 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:33:58.173476 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:33:58.173606 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:33:58.174264 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:33:58.174267 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:33:58.174269 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:33:58.174628 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:33:58.174839 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0810 11:33:58.175193 (+ 354us) server_negotiation.cc:299] Negotiation successful
0810 11:33:58.175384 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260810 11:33:58.180397 22555 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48892:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 11:33:58.188299 22555 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 11:33:58.218834 23034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:33:58.203931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.194:44985 (local address 127.0.0.1:43448)
0810 11:33:58.204616 (+ 685us) negotiation.cc:107] Waiting for socket to connect
0810 11:33:58.204631 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 11:33:58.204774 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:33:58.206934 (+ 2160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:33:58.206946 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:33:58.206997 (+ 51us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:33:58.207705 (+ 708us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:33:58.207725 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:33:58.210442 (+ 2717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:33:58.210453 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:33:58.214265 (+ 3812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:33:58.214297 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:33:58.217490 (+ 3193us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:33:58.217499 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:33:58.217829 (+ 330us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:33:58.217899 (+ 70us) client_negotiation.cc:769] Sending connection context
0810 11:33:58.218062 (+ 163us) client_negotiation.cc:240] Negotiation successful
0810 11:33:58.218318 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":556,"mutex_wait_us":60,"thread_start_us":444,"threads_started":1}
I20260810 11:33:58.220050 23035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:33:58.205042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43448 (local address 127.8.18.194:44985)
0810 11:33:58.205752 (+ 710us) server_negotiation.cc:206] Beginning negotiation
0810 11:33:58.205765 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 11:33:58.205805 (+ 40us) server_negotiation.cc:407] Connection header received
0810 11:33:58.205933 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:33:58.205944 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:33:58.206267 (+ 323us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:33:58.206529 (+ 262us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:33:58.208073 (+ 1544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:33:58.210125 (+ 2052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:33:58.214659 (+ 4534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:33:58.216828 (+ 2169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:33:58.217024 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:33:58.217285 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0810 11:33:58.219423 (+ 2138us) server_negotiation.cc:299] Negotiation successful
0810 11:33:58.219684 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":526,"thread_start_us":384,"threads_started":1}
I20260810 11:33:58.220715 23033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:33:58.202530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.193:41937 (local address 127.0.0.1:45634)
0810 11:33:58.204191 (+ 1661us) negotiation.cc:107] Waiting for socket to connect
0810 11:33:58.204249 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0810 11:33:58.204587 (+ 338us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:33:58.207635 (+ 3048us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:33:58.207643 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:33:58.207691 (+ 48us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:33:58.208337 (+ 646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:33:58.208359 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:33:58.213514 (+ 5155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:33:58.213524 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:33:58.217833 (+ 4309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:33:58.217851 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:33:58.219857 (+ 2006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:33:58.219862 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:33:58.219997 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:33:58.220020 (+ 23us) client_negotiation.cc:769] Sending connection context
0810 11:33:58.220137 (+ 117us) client_negotiation.cc:240] Negotiation successful
0810 11:33:58.220311 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1361,"mutex_wait_us":2,"thread_start_us":1140,"threads_started":1}
I20260810 11:33:58.221244 23032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:33:58.202551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45634 (local address 127.8.18.193:41937)
0810 11:33:58.203465 (+ 914us) server_negotiation.cc:206] Beginning negotiation
0810 11:33:58.203479 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 11:33:58.204388 (+ 909us) server_negotiation.cc:407] Connection header received
0810 11:33:58.204881 (+ 493us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:33:58.204890 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:33:58.205197 (+ 307us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:33:58.205498 (+ 301us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:33:58.208673 (+ 3175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:33:58.211105 (+ 2432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:33:58.218087 (+ 6982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:33:58.219417 (+ 1330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:33:58.219519 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:33:58.219696 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0810 11:33:58.220424 (+ 728us) server_negotiation.cc:299] Negotiation successful
0810 11:33:58.220616 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":626,"thread_start_us":378,"threads_started":1}
I20260810 11:33:58.224761 22818 tablet_service.cc:1467] Processing CreateTablet for tablet f5f19a8c9997484296723380c23dcf96 (DEFAULT_TABLE table=test-table [id=47208c35e7054c25a15c2e048f11452a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:33:58.226286 22818 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f5f19a8c9997484296723380c23dcf96. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:33:58.227144 23036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:33:58.204652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.195:43839 (local address 127.0.0.1:34810)
0810 11:33:58.216163 (+ 11511us) negotiation.cc:107] Waiting for socket to connect
0810 11:33:58.216190 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0810 11:33:58.216380 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:33:58.218111 (+ 1731us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:33:58.218116 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:33:58.218144 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:33:58.218566 (+ 422us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:33:58.218579 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:33:58.221174 (+ 2595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:33:58.221180 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:33:58.223114 (+ 1934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:33:58.223129 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:33:58.226182 (+ 3053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:33:58.226191 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:33:58.226423 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:33:58.226460 (+ 37us) client_negotiation.cc:769] Sending connection context
0810 11:33:58.226589 (+ 129us) client_negotiation.cc:240] Negotiation successful
0810 11:33:58.226756 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11318,"thread_start_us":11853,"threads_started":1}
I20260810 11:33:58.227180 23037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:33:58.205315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34810 (local address 127.8.18.195:43839)
0810 11:33:58.217310 (+ 11995us) server_negotiation.cc:206] Beginning negotiation
0810 11:33:58.217322 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 11:33:58.217361 (+ 39us) server_negotiation.cc:407] Connection header received
0810 11:33:58.217502 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:33:58.217522 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:33:58.217843 (+ 321us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:33:58.218812 (+ 969us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:33:58.219165 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:33:58.220810 (+ 1645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:33:58.224654 (+ 3844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:33:58.225923 (+ 1269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:33:58.226107 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:33:58.226376 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0810 11:33:58.226644 (+ 268us) server_negotiation.cc:299] Negotiation successful
0810 11:33:58.226836 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11790,"thread_start_us":11608,"threads_started":1}
I20260810 11:33:58.230804 22954 tablet_service.cc:1467] Processing CreateTablet for tablet f5f19a8c9997484296723380c23dcf96 (DEFAULT_TABLE table=test-table [id=47208c35e7054c25a15c2e048f11452a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:33:58.232237 22954 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f5f19a8c9997484296723380c23dcf96. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:33:58.232148 22682 tablet_service.cc:1467] Processing CreateTablet for tablet f5f19a8c9997484296723380c23dcf96 (DEFAULT_TABLE table=test-table [id=47208c35e7054c25a15c2e048f11452a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:33:58.233889 22682 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f5f19a8c9997484296723380c23dcf96. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:33:58.245517 23038 tablet_bootstrap.cc:492] T f5f19a8c9997484296723380c23dcf96 P abbaeefbabb941d8ae31fe6e637fdbc0: Bootstrap starting.
I20260810 11:33:58.247867 23039 tablet_bootstrap.cc:492] T f5f19a8c9997484296723380c23dcf96 P 52042505dd7f42e990ee5684ada73014: Bootstrap starting.
I20260810 11:33:58.250172 23038 tablet_bootstrap.cc:654] T f5f19a8c9997484296723380c23dcf96 P abbaeefbabb941d8ae31fe6e637fdbc0: Neither blocks nor log segments found. Creating new log.
I20260810 11:33:58.251658 23040 tablet_bootstrap.cc:492] T f5f19a8c9997484296723380c23dcf96 P b53ea98f7b4b4a828f90fc4578efe84a: Bootstrap starting.
I20260810 11:33:58.251847 23038 log.cc:824] T f5f19a8c9997484296723380c23dcf96 P abbaeefbabb941d8ae31fe6e637fdbc0: Log is configured to *not* fsync() on all Append() calls
I20260810 11:33:58.252619 23039 tablet_bootstrap.cc:654] T f5f19a8c9997484296723380c23dcf96 P 52042505dd7f42e990ee5684ada73014: Neither blocks nor log segments found. Creating new log.
I20260810 11:33:58.254273 23039 log.cc:824] T f5f19a8c9997484296723380c23dcf96 P 52042505dd7f42e990ee5684ada73014: Log is configured to *not* fsync() on all Append() calls
I20260810 11:33:58.256395 23040 tablet_bootstrap.cc:654] T f5f19a8c9997484296723380c23dcf96 P b53ea98f7b4b4a828f90fc4578efe84a: Neither blocks nor log segments found. Creating new log.
I20260810 11:33:58.256832 23038 tablet_bootstrap.cc:492] T f5f19a8c9997484296723380c23dcf96 P abbaeefbabb941d8ae31fe6e637fdbc0: No bootstrap required, opened a new log
I20260810 11:33:58.257277 23038 ts_tablet_manager.cc:1397] T f5f19a8c9997484296723380c23dcf96 P abbaeefbabb941d8ae31fe6e637fdbc0: Time spent bootstrapping tablet: real 0.012s user 0.007s sys 0.001s
I20260810 11:33:58.257825 23040 log.cc:824] T f5f19a8c9997484296723380c23dcf96 P b53ea98f7b4b4a828f90fc4578efe84a: Log is configured to *not* fsync() on all Append() calls
I20260810 11:33:58.258119 23039 tablet_bootstrap.cc:492] T f5f19a8c9997484296723380c23dcf96 P 52042505dd7f42e990ee5684ada73014: No bootstrap required, opened a new log
I20260810 11:33:58.258494 23039 ts_tablet_manager.cc:1397] T f5f19a8c9997484296723380c23dcf96 P 52042505dd7f42e990ee5684ada73014: Time spent bootstrapping tablet: real 0.011s user 0.009s sys 0.000s
I20260810 11:33:58.261379 23040 tablet_bootstrap.cc:492] T f5f19a8c9997484296723380c23dcf96 P b53ea98f7b4b4a828f90fc4578efe84a: No bootstrap required, opened a new log
I20260810 11:33:58.261826 23040 ts_tablet_manager.cc:1397] T f5f19a8c9997484296723380c23dcf96 P b53ea98f7b4b4a828f90fc4578efe84a: Time spent bootstrapping tablet: real 0.011s user 0.007s sys 0.002s
I20260810 11:33:58.269645 23038 raft_consensus.cc:348] T f5f19a8c9997484296723380c23dcf96 P abbaeefbabb941d8ae31fe6e637fdbc0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b53ea98f7b4b4a828f90fc4578efe84a" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 41937 } } peers { permanent_uuid: "52042505dd7f42e990ee5684ada73014" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 43839 } } peers { permanent_uuid: "abbaeefbabb941d8ae31fe6e637fdbc0" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 44985 } }
I20260810 11:33:58.270280 23038 raft_consensus.cc:374] T f5f19a8c9997484296723380c23dcf96 P abbaeefbabb941d8ae31fe6e637fdbc0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:33:58.270399 23038 raft_consensus.cc:729] T f5f19a8c9997484296723380c23dcf96 P abbaeefbabb941d8ae31fe6e637fdbc0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abbaeefbabb941d8ae31fe6e637fdbc0, State: Initialized, Role: FOLLOWER
I20260810 11:33:58.270273 23040 raft_consensus.cc:348] T f5f19a8c9997484296723380c23dcf96 P b53ea98f7b4b4a828f90fc4578efe84a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b53ea98f7b4b4a828f90fc4578efe84a" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 41937 } } peers { permanent_uuid: "52042505dd7f42e990ee5684ada73014" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 43839 } } peers { permanent_uuid: "abbaeefbabb941d8ae31fe6e637fdbc0" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 44985 } }
I20260810 11:33:58.270764 23040 raft_consensus.cc:374] T f5f19a8c9997484296723380c23dcf96 P b53ea98f7b4b4a828f90fc4578efe84a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:33:58.270474 23039 raft_consensus.cc:348] T f5f19a8c9997484296723380c23dcf96 P 52042505dd7f42e990ee5684ada73014 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b53ea98f7b4b4a828f90fc4578efe84a" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 41937 } } peers { permanent_uuid: "52042505dd7f42e990ee5684ada73014" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 43839 } } peers { permanent_uuid: "abbaeefbabb941d8ae31fe6e637fdbc0" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 44985 } }
I20260810 11:33:58.270898 23040 raft_consensus.cc:729] T f5f19a8c9997484296723380c23dcf96 P b53ea98f7b4b4a828f90fc4578efe84a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b53ea98f7b4b4a828f90fc4578efe84a, State: Initialized, Role: FOLLOWER
I20260810 11:33:58.270941 23039 raft_consensus.cc:374] T f5f19a8c9997484296723380c23dcf96 P 52042505dd7f42e990ee5684ada73014 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:33:58.271034 23039 raft_consensus.cc:729] T f5f19a8c9997484296723380c23dcf96 P 52042505dd7f42e990ee5684ada73014 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52042505dd7f42e990ee5684ada73014, State: Initialized, Role: FOLLOWER
I20260810 11:33:58.271320 23038 consensus_queue.cc:260] T f5f19a8c9997484296723380c23dcf96 P abbaeefbabb941d8ae31fe6e637fdbc0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b53ea98f7b4b4a828f90fc4578efe84a" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 41937 } } peers { permanent_uuid: "52042505dd7f42e990ee5684ada73014" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 43839 } } peers { permanent_uuid: "abbaeefbabb941d8ae31fe6e637fdbc0" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 44985 } }
I20260810 11:33:58.271507 23040 consensus_queue.cc:260] T f5f19a8c9997484296723380c23dcf96 P b53ea98f7b4b4a828f90fc4578efe84a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b53ea98f7b4b4a828f90fc4578efe84a" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 41937 } } peers { permanent_uuid: "52042505dd7f42e990ee5684ada73014" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 43839 } } peers { permanent_uuid: "abbaeefbabb941d8ae31fe6e637fdbc0" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 44985 } }
I20260810 11:33:58.271776 23039 consensus_queue.cc:260] T f5f19a8c9997484296723380c23dcf96 P 52042505dd7f42e990ee5684ada73014 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b53ea98f7b4b4a828f90fc4578efe84a" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 41937 } } peers { permanent_uuid: "52042505dd7f42e990ee5684ada73014" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 43839 } } peers { permanent_uuid: "abbaeefbabb941d8ae31fe6e637fdbc0" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 44985 } }
I20260810 11:33:58.272948 22883 heartbeater.cc:503] Master 127.8.18.254:43799 was elected leader, sending a full tablet report...
I20260810 11:33:58.273978 23038 ts_tablet_manager.cc:1428] T f5f19a8c9997484296723380c23dcf96 P abbaeefbabb941d8ae31fe6e637fdbc0: Time spent starting tablet: real 0.017s user 0.012s sys 0.006s
I20260810 11:33:58.276500 23019 heartbeater.cc:503] Master 127.8.18.254:43799 was elected leader, sending a full tablet report...
I20260810 11:33:58.276913 23040 ts_tablet_manager.cc:1428] T f5f19a8c9997484296723380c23dcf96 P b53ea98f7b4b4a828f90fc4578efe84a: Time spent starting tablet: real 0.015s user 0.013s sys 0.002s
I20260810 11:33:58.280207 23039 ts_tablet_manager.cc:1428] T f5f19a8c9997484296723380c23dcf96 P 52042505dd7f42e990ee5684ada73014: Time spent starting tablet: real 0.022s user 0.014s sys 0.003s
I20260810 11:33:58.367306 23045 raft_consensus.cc:482] T f5f19a8c9997484296723380c23dcf96 P b53ea98f7b4b4a828f90fc4578efe84a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 11:33:58.367734 23045 raft_consensus.cc:504] T f5f19a8c9997484296723380c23dcf96 P b53ea98f7b4b4a828f90fc4578efe84a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b53ea98f7b4b4a828f90fc4578efe84a" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 41937 } } peers { permanent_uuid: "52042505dd7f42e990ee5684ada73014" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 43839 } } peers { permanent_uuid: "abbaeefbabb941d8ae31fe6e637fdbc0" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 44985 } }
W20260810 11:33:58.368479 23020 tablet.cc:2369] T f5f19a8c9997484296723380c23dcf96 P 52042505dd7f42e990ee5684ada73014: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 11:33:58.369848 23045 leader_election.cc:288] T f5f19a8c9997484296723380c23dcf96 P b53ea98f7b4b4a828f90fc4578efe84a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 52042505dd7f42e990ee5684ada73014 (127.8.18.195:43839), abbaeefbabb941d8ae31fe6e637fdbc0 (127.8.18.194:44985)
I20260810 11:33:58.381507 23035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:33:58.370784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.193:36139 (local address 127.8.18.194:44985)
0810 11:33:58.371080 (+ 296us) server_negotiation.cc:206] Beginning negotiation
0810 11:33:58.371089 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 11:33:58.372270 (+ 1181us) server_negotiation.cc:407] Connection header received
0810 11:33:58.372430 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:33:58.372445 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:33:58.372528 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:33:58.372635 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:33:58.373481 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:33:58.374671 (+ 1190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:33:58.378606 (+ 3935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:33:58.379963 (+ 1357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:33:58.380233 (+ 270us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:33:58.380302 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0810 11:33:58.381031 (+ 729us) server_negotiation.cc:299] Negotiation successful
0810 11:33:58.381227 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260810 11:33:58.381507 23037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:33:58.370392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.193:55705 (local address 127.8.18.195:43839)
0810 11:33:58.370967 (+ 575us) server_negotiation.cc:206] Beginning negotiation
0810 11:33:58.370975 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:33:58.371888 (+ 913us) server_negotiation.cc:407] Connection header received
0810 11:33:58.372214 (+ 326us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:33:58.372220 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:33:58.372309 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:33:58.372435 (+ 126us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:33:58.373481 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:33:58.374671 (+ 1190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:33:58.378606 (+ 3935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:33:58.379963 (+ 1357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:33:58.380188 (+ 225us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:33:58.380273 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0810 11:33:58.381031 (+ 758us) server_negotiation.cc:299] Negotiation successful
0810 11:33:58.381185 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":409}
I20260810 11:33:58.381867 23048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:33:58.370255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.195:43839 (local address 127.8.18.193:55705)
0810 11:33:58.371750 (+ 1495us) negotiation.cc:107] Waiting for socket to connect
0810 11:33:58.371776 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0810 11:33:58.372033 (+ 257us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:33:58.372557 (+ 524us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:33:58.372561 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:33:58.372630 (+ 69us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:33:58.373233 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:33:58.373255 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:33:58.374870 (+ 1615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:33:58.374877 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:33:58.378300 (+ 3423us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:33:58.378328 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:33:58.380384 (+ 2056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:33:58.380393 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:33:58.380719 (+ 326us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:33:58.380842 (+ 123us) client_negotiation.cc:769] Sending connection context
0810 11:33:58.380970 (+ 128us) client_negotiation.cc:240] Negotiation successful
0810 11:33:58.381260 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1297,"mutex_wait_us":321,"thread_start_us":1477,"threads_started":1}
I20260810 11:33:58.381891 23047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:33:58.370496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.194:44985 (local address 127.8.18.193:36139)
0810 11:33:58.372139 (+ 1643us) negotiation.cc:107] Waiting for socket to connect
0810 11:33:58.372159 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 11:33:58.372294 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:33:58.372727 (+ 433us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:33:58.372735 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:33:58.372782 (+ 47us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:33:58.373259 (+ 477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:33:58.373270 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:33:58.374870 (+ 1600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:33:58.374877 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:33:58.378300 (+ 3423us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:33:58.378328 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:33:58.380398 (+ 2070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:33:58.380404 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:33:58.380726 (+ 322us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:33:58.380840 (+ 114us) client_negotiation.cc:769] Sending connection context
0810 11:33:58.380970 (+ 130us) client_negotiation.cc:240] Negotiation successful
0810 11:33:58.381221 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1501,"mutex_wait_us":388,"thread_start_us":802,"threads_started":1}
I20260810 11:33:58.383373 22974 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5f19a8c9997484296723380c23dcf96" candidate_uuid: "b53ea98f7b4b4a828f90fc4578efe84a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52042505dd7f42e990ee5684ada73014" is_pre_election: true
I20260810 11:33:58.383742 22838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5f19a8c9997484296723380c23dcf96" candidate_uuid: "b53ea98f7b4b4a828f90fc4578efe84a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abbaeefbabb941d8ae31fe6e637fdbc0" is_pre_election: true
I20260810 11:33:58.384126 22974 raft_consensus.cc:2454] T f5f19a8c9997484296723380c23dcf96 P 52042505dd7f42e990ee5684ada73014 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b53ea98f7b4b4a828f90fc4578efe84a in term 0.
I20260810 11:33:58.384371 22838 raft_consensus.cc:2454] T f5f19a8c9997484296723380c23dcf96 P abbaeefbabb941d8ae31fe6e637fdbc0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b53ea98f7b4b4a828f90fc4578efe84a in term 0.
I20260810 11:33:58.385157 22636 leader_election.cc:302] T f5f19a8c9997484296723380c23dcf96 P b53ea98f7b4b4a828f90fc4578efe84a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52042505dd7f42e990ee5684ada73014, b53ea98f7b4b4a828f90fc4578efe84a; no voters:
I20260810 11:33:58.385754 23045 raft_consensus.cc:2781] T f5f19a8c9997484296723380c23dcf96 P b53ea98f7b4b4a828f90fc4578efe84a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 11:33:58.385941 23045 raft_consensus.cc:482] T f5f19a8c9997484296723380c23dcf96 P b53ea98f7b4b4a828f90fc4578efe84a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 11:33:58.386133 23045 raft_consensus.cc:3037] T f5f19a8c9997484296723380c23dcf96 P b53ea98f7b4b4a828f90fc4578efe84a [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:33:58.388669 23045 raft_consensus.cc:504] T f5f19a8c9997484296723380c23dcf96 P b53ea98f7b4b4a828f90fc4578efe84a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b53ea98f7b4b4a828f90fc4578efe84a" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 41937 } } peers { permanent_uuid: "52042505dd7f42e990ee5684ada73014" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 43839 } } peers { permanent_uuid: "abbaeefbabb941d8ae31fe6e637fdbc0" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 44985 } }
I20260810 11:33:58.389600 23045 leader_election.cc:288] T f5f19a8c9997484296723380c23dcf96 P b53ea98f7b4b4a828f90fc4578efe84a [CANDIDATE]: Term 1 election: Requested vote from peers 52042505dd7f42e990ee5684ada73014 (127.8.18.195:43839), abbaeefbabb941d8ae31fe6e637fdbc0 (127.8.18.194:44985)
I20260810 11:33:58.390377 22974 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5f19a8c9997484296723380c23dcf96" candidate_uuid: "b53ea98f7b4b4a828f90fc4578efe84a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52042505dd7f42e990ee5684ada73014"
I20260810 11:33:58.390363 22838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5f19a8c9997484296723380c23dcf96" candidate_uuid: "b53ea98f7b4b4a828f90fc4578efe84a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abbaeefbabb941d8ae31fe6e637fdbc0"
I20260810 11:33:58.390648 22974 raft_consensus.cc:3037] T f5f19a8c9997484296723380c23dcf96 P 52042505dd7f42e990ee5684ada73014 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:33:58.390648 22838 raft_consensus.cc:3037] T f5f19a8c9997484296723380c23dcf96 P abbaeefbabb941d8ae31fe6e637fdbc0 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:33:58.394384 22838 raft_consensus.cc:2454] T f5f19a8c9997484296723380c23dcf96 P abbaeefbabb941d8ae31fe6e637fdbc0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b53ea98f7b4b4a828f90fc4578efe84a in term 1.
I20260810 11:33:58.394384 22974 raft_consensus.cc:2454] T f5f19a8c9997484296723380c23dcf96 P 52042505dd7f42e990ee5684ada73014 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b53ea98f7b4b4a828f90fc4578efe84a in term 1.
I20260810 11:33:58.395035 22636 leader_election.cc:302] T f5f19a8c9997484296723380c23dcf96 P b53ea98f7b4b4a828f90fc4578efe84a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52042505dd7f42e990ee5684ada73014, b53ea98f7b4b4a828f90fc4578efe84a; no voters:
I20260810 11:33:58.395442 23045 raft_consensus.cc:2781] T f5f19a8c9997484296723380c23dcf96 P b53ea98f7b4b4a828f90fc4578efe84a [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:33:58.396032 23045 raft_consensus.cc:686] T f5f19a8c9997484296723380c23dcf96 P b53ea98f7b4b4a828f90fc4578efe84a [term 1 LEADER]: Becoming Leader. State: Replica: b53ea98f7b4b4a828f90fc4578efe84a, State: Running, Role: LEADER
I20260810 11:33:58.396660 23045 consensus_queue.cc:237] T f5f19a8c9997484296723380c23dcf96 P b53ea98f7b4b4a828f90fc4578efe84a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b53ea98f7b4b4a828f90fc4578efe84a" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 41937 } } peers { permanent_uuid: "52042505dd7f42e990ee5684ada73014" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 43839 } } peers { permanent_uuid: "abbaeefbabb941d8ae31fe6e637fdbc0" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 44985 } }
W20260810 11:33:58.400370 22748 tablet.cc:2369] T f5f19a8c9997484296723380c23dcf96 P b53ea98f7b4b4a828f90fc4578efe84a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 11:33:58.402033 22552 catalog_manager.cc:5495] T f5f19a8c9997484296723380c23dcf96 P b53ea98f7b4b4a828f90fc4578efe84a reported cstate change: term changed from 0 to 1, leader changed from <none> to b53ea98f7b4b4a828f90fc4578efe84a (127.8.18.193). New cstate: current_term: 1 leader_uuid: "b53ea98f7b4b4a828f90fc4578efe84a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b53ea98f7b4b4a828f90fc4578efe84a" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 41937 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52042505dd7f42e990ee5684ada73014" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 43839 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "abbaeefbabb941d8ae31fe6e637fdbc0" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 44985 } health_report { overall_health: UNKNOWN } } }
I20260810 11:33:58.424517 23032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:33:58.418107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45640 (local address 127.8.18.193:41937)
0810 11:33:58.418342 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0810 11:33:58.418347 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:33:58.418367 (+ 20us) server_negotiation.cc:407] Connection header received
0810 11:33:58.418545 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:33:58.418548 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:33:58.418633 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:33:58.418777 (+ 144us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 11:33:58.419262 (+ 485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:33:58.419991 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:33:58.421667 (+ 1676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:33:58.422125 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:33:58.422242 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:33:58.423013 (+ 771us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 11:33:58.423703 (+ 690us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 11:33:58.423856 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0810 11:33:58.424055 (+ 199us) server_negotiation.cc:299] Negotiation successful
0810 11:33:58.424138 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260810 11:33:58.433892 22974 raft_consensus.cc:1261] T f5f19a8c9997484296723380c23dcf96 P 52042505dd7f42e990ee5684ada73014 [term 1 FOLLOWER]: Refusing update from remote peer b53ea98f7b4b4a828f90fc4578efe84a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 11:33:58.433892 22838 raft_consensus.cc:1261] T f5f19a8c9997484296723380c23dcf96 P abbaeefbabb941d8ae31fe6e637fdbc0 [term 1 FOLLOWER]: Refusing update from remote peer b53ea98f7b4b4a828f90fc4578efe84a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 11:33:58.435133 23049 consensus_queue.cc:1035] T f5f19a8c9997484296723380c23dcf96 P b53ea98f7b4b4a828f90fc4578efe84a [LEADER]: Connected to new peer: Peer: permanent_uuid: "abbaeefbabb941d8ae31fe6e637fdbc0" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 44985 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 11:33:58.435600 23045 consensus_queue.cc:1035] T f5f19a8c9997484296723380c23dcf96 P b53ea98f7b4b4a828f90fc4578efe84a [LEADER]: Connected to new peer: Peer: permanent_uuid: "52042505dd7f42e990ee5684ada73014" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 43839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 11:33:58.449488 23052 mvcc.cc:205] Tried to move back new op lower bound from 7316937271013736448 to 7316937270877184000. Current Snapshot: MvccSnapshot[applied={T|T < 7316937271013736448}]
I20260810 11:33:58.464970 22553 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48892:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 11:33:58.465327 22553 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48892:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 11:33:58.469503 22553 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 99954a220cf4425b894a741dd117548a: Sending DeleteTablet for 3 replicas of tablet f5f19a8c9997484296723380c23dcf96
I20260810 11:33:58.471100 22682 tablet_service.cc:1514] Processing DeleteTablet for tablet f5f19a8c9997484296723380c23dcf96 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:33:58 UTC) from {username='kudu', principal='kudu/127.8.18.254@KRBTEST.COM'} at 127.0.0.1:45634
I20260810 11:33:58.471148 22818 tablet_service.cc:1514] Processing DeleteTablet for tablet f5f19a8c9997484296723380c23dcf96 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:33:58 UTC) from {username='kudu', principal='kudu/127.8.18.254@KRBTEST.COM'} at 127.0.0.1:43448
I20260810 11:33:58.471256 22954 tablet_service.cc:1514] Processing DeleteTablet for tablet f5f19a8c9997484296723380c23dcf96 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:33:58 UTC) from {username='kudu', principal='kudu/127.8.18.254@KRBTEST.COM'} at 127.0.0.1:34810
I20260810 11:33:58.472546 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 22617
I20260810 11:33:58.521646 23065 tablet_replica.cc:330] T f5f19a8c9997484296723380c23dcf96 P 52042505dd7f42e990ee5684ada73014: stopping tablet replica
I20260810 11:33:58.522053 23063 tablet_replica.cc:330] T f5f19a8c9997484296723380c23dcf96 P b53ea98f7b4b4a828f90fc4578efe84a: stopping tablet replica
I20260810 11:33:58.522614 23063 raft_consensus.cc:2229] T f5f19a8c9997484296723380c23dcf96 P b53ea98f7b4b4a828f90fc4578efe84a [term 1 LEADER]: Raft consensus shutting down.
I20260810 11:33:58.522671 23065 raft_consensus.cc:2229] T f5f19a8c9997484296723380c23dcf96 P 52042505dd7f42e990ee5684ada73014 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 11:33:58.523458 23063 raft_consensus.cc:2258] T f5f19a8c9997484296723380c23dcf96 P b53ea98f7b4b4a828f90fc4578efe84a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:33:58.523691 23065 raft_consensus.cc:2258] T f5f19a8c9997484296723380c23dcf96 P 52042505dd7f42e990ee5684ada73014 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:33:58.525787 23062 tablet_replica.cc:330] T f5f19a8c9997484296723380c23dcf96 P abbaeefbabb941d8ae31fe6e637fdbc0: stopping tablet replica
I20260810 11:33:58.526304 23062 raft_consensus.cc:2229] T f5f19a8c9997484296723380c23dcf96 P abbaeefbabb941d8ae31fe6e637fdbc0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 11:33:58.526995 23062 raft_consensus.cc:2258] T f5f19a8c9997484296723380c23dcf96 P abbaeefbabb941d8ae31fe6e637fdbc0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:33:58.528435 23065 ts_tablet_manager.cc:1905] T f5f19a8c9997484296723380c23dcf96 P 52042505dd7f42e990ee5684ada73014: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:33:58.529191 23062 ts_tablet_manager.cc:1905] T f5f19a8c9997484296723380c23dcf96 P abbaeefbabb941d8ae31fe6e637fdbc0: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 10 11:33:58 dist-test-slave-srgv krb5kdc[22509](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361635, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.18.193@KRBTEST.COM
I20260810 11:33:58.537472 23063 ts_tablet_manager.cc:1905] T f5f19a8c9997484296723380c23dcf96 P b53ea98f7b4b4a828f90fc4578efe84a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:33:58.546128 23063 ts_tablet_manager.cc:1918] T f5f19a8c9997484296723380c23dcf96 P b53ea98f7b4b4a828f90fc4578efe84a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 11:33:58.546340 23063 log.cc:1196] T f5f19a8c9997484296723380c23dcf96 P b53ea98f7b4b4a828f90fc4578efe84a: Deleting WAL directory at /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/ts-0/wal/wals/f5f19a8c9997484296723380c23dcf96
I20260810 11:33:58.546995 23063 ts_tablet_manager.cc:1939] T f5f19a8c9997484296723380c23dcf96 P b53ea98f7b4b4a828f90fc4578efe84a: Deleting consensus metadata
I20260810 11:33:58.547573 23032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:33:58.474366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45642 (local address 127.8.18.193:41937)
0810 11:33:58.474635 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0810 11:33:58.474645 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 11:33:58.496107 (+ 21462us) server_negotiation.cc:407] Connection header received
0810 11:33:58.496805 (+ 698us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:33:58.496815 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:33:58.496946 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:33:58.497060 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:33:58.500526 (+ 3466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:33:58.506705 (+ 6179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:33:58.525213 (+ 18508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:33:58.526064 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:33:58.526222 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:33:58.538222 (+ 12000us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:33:58.538244 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:33:58.538265 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:33:58.538326 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:33:58.542911 (+ 4585us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:33:58.544137 (+ 1226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:33:58.544151 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:33:58.544155 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:33:58.544386 (+ 231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:33:58.544925 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:33:58.544931 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:33:58.544934 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:33:58.545494 (+ 560us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:33:58.545810 (+ 316us) server_negotiation.cc:1035] Waiting for connection context
0810 11:33:58.546748 (+ 938us) server_negotiation.cc:299] Negotiation successful
0810 11:33:58.547170 (+ 422us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260810 11:33:58.548447 23065 ts_tablet_manager.cc:1918] T f5f19a8c9997484296723380c23dcf96 P 52042505dd7f42e990ee5684ada73014: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 11:33:58.548661 23065 log.cc:1196] T f5f19a8c9997484296723380c23dcf96 P 52042505dd7f42e990ee5684ada73014: Deleting WAL directory at /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/ts-2/wal/wals/f5f19a8c9997484296723380c23dcf96
I20260810 11:33:58.549153 22541 catalog_manager.cc:4841] TS b53ea98f7b4b4a828f90fc4578efe84a (127.8.18.193:41937): tablet f5f19a8c9997484296723380c23dcf96 (table test-table [id=47208c35e7054c25a15c2e048f11452a]) successfully deleted
I20260810 11:33:58.549244 23065 ts_tablet_manager.cc:1939] T f5f19a8c9997484296723380c23dcf96 P 52042505dd7f42e990ee5684ada73014: Deleting consensus metadata
I20260810 11:33:58.549819 22742 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:33:58.549862 23062 ts_tablet_manager.cc:1918] T f5f19a8c9997484296723380c23dcf96 P abbaeefbabb941d8ae31fe6e637fdbc0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 11:33:58.550134 23062 log.cc:1196] T f5f19a8c9997484296723380c23dcf96 P abbaeefbabb941d8ae31fe6e637fdbc0: Deleting WAL directory at /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/ts-1/wal/wals/f5f19a8c9997484296723380c23dcf96
I20260810 11:33:58.550751 23062 ts_tablet_manager.cc:1939] T f5f19a8c9997484296723380c23dcf96 P abbaeefbabb941d8ae31fe6e637fdbc0: Deleting consensus metadata
I20260810 11:33:58.551398 22541 catalog_manager.cc:4841] TS 52042505dd7f42e990ee5684ada73014 (127.8.18.195:43839): tablet f5f19a8c9997484296723380c23dcf96 (table test-table [id=47208c35e7054c25a15c2e048f11452a]) successfully deleted
I20260810 11:33:58.553278 22542 catalog_manager.cc:4841] TS abbaeefbabb941d8ae31fe6e637fdbc0 (127.8.18.194:44985): tablet f5f19a8c9997484296723380c23dcf96 (table test-table [id=47208c35e7054c25a15c2e048f11452a]) successfully deleted
I20260810 11:33:58.658951 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 22617
I20260810 11:33:58.681996 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 22753
Aug 10 11:33:58 dist-test-slave-srgv krb5kdc[22509](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361635, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.18.194@KRBTEST.COM
I20260810 11:33:58.704352 23035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:33:58.682932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43462 (local address 127.8.18.194:44985)
0810 11:33:58.683125 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0810 11:33:58.683130 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:33:58.683148 (+ 18us) server_negotiation.cc:407] Connection header received
0810 11:33:58.683241 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:33:58.683246 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:33:58.683332 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:33:58.683429 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:33:58.684779 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:33:58.685618 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:33:58.687115 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:33:58.687528 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:33:58.687633 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:33:58.697699 (+ 10066us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:33:58.697724 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:33:58.697742 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:33:58.697815 (+ 73us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:33:58.701041 (+ 3226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:33:58.702245 (+ 1204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:33:58.702254 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:33:58.702256 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:33:58.702412 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:33:58.702773 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:33:58.702776 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:33:58.702778 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:33:58.703234 (+ 456us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:33:58.703495 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0810 11:33:58.703743 (+ 248us) server_negotiation.cc:299] Negotiation successful
0810 11:33:58.703936 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260810 11:33:58.705132 22878 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:33:58.806676 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 22753
I20260810 11:33:58.827162 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 22889
Aug 10 11:33:58 dist-test-slave-srgv krb5kdc[22509](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361635, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.18.195@KRBTEST.COM
I20260810 11:33:58.850989 23037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:33:58.828040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34812 (local address 127.8.18.195:43839)
0810 11:33:58.828268 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0810 11:33:58.828277 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 11:33:58.828301 (+ 24us) server_negotiation.cc:407] Connection header received
0810 11:33:58.828380 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:33:58.828384 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:33:58.828448 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:33:58.828583 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:33:58.830006 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:33:58.830735 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:33:58.832061 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:33:58.832468 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:33:58.832622 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:33:58.843581 (+ 10959us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:33:58.843602 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:33:58.843620 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:33:58.843667 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:33:58.847250 (+ 3583us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:33:58.848273 (+ 1023us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:33:58.848283 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:33:58.848286 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:33:58.848482 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:33:58.848962 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:33:58.848968 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:33:58.848974 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:33:58.849471 (+ 497us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:33:58.849831 (+ 360us) server_negotiation.cc:1035] Waiting for connection context
0810 11:33:58.850264 (+ 433us) server_negotiation.cc:299] Negotiation successful
0810 11:33:58.850548 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260810 11:33:58.851895 23014 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:33:58.947850 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 22889
I20260810 11:33:58.970168 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 22522
I20260810 11:33:58.971541 22586 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:33:59.096199 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 22522
I20260810 11:33:59.122224 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.18.254:43799
--webserver_interface=127.8.18.254
--webserver_port=40949
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.254.keytab
--principal=kudu/127.8.18.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.18.212:35253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.18.254:43799
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:33:59.509914 23072 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:33:59.510478 23072 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:33:59.510689 23072 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:33:59.519605 23072 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 11:33:59.519753 23072 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:33:59.519812 23072 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:33:59.519866 23072 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 11:33:59.519913 23072 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 11:33:59.531538 23072 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.18.212:35253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.8.18.254:43799
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.254.keytab
--principal=kudu/127.8.18.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.18.254:43799
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.18.254
--webserver_port=40949
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:33:59.532997 23072 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:33:59.534617 23072 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:33:59.563073 23078 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:33:59.563048 23079 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:33:59.563006 23081 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:33:59.564328 23072 server_base.cc:1034] running on GCE node
I20260810 11:33:59.565383 23072 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:33:59.567379 23072 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:33:59.568698 23072 hybrid_clock.cc:648] HybridClock initialized: now 1786361639568644 us; error 44 us; skew 500 ppm
I20260810 11:33:59.569284 23072 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:33:59 dist-test-slave-srgv krb5kdc[22509](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361639, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:33:59.574486 23072 init.cc:377] Logged in from keytab as kudu/127.8.18.254@KRBTEST.COM (short username kudu)
I20260810 11:33:59.576859 23072 webserver.cc:460] Webserver started at http://127.8.18.254:40949/ using document root <none> and password file <none>
I20260810 11:33:59.577944 23072 fs_manager.cc:359] Metadata directory not provided
I20260810 11:33:59.578157 23072 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:33:59.586010 23072 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.007s sys 0.000s
I20260810 11:33:59.590034 23089 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:33:59.591938 23072 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260810 11:33:59.592228 23072 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/master-0/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/master-0/wal
uuid: "99954a220cf4425b894a741dd117548a"
format_stamp: "Formatted at 2026-08-10 11:33:56 on dist-test-slave-srgv"
server_key: "dd0a6aa35d36a6fdd867b117088c8f9a"
server_key_iv: "6239f1ab4e6711ccf45bd557db90dd00"
server_key_version: "encryptionkey@0"
I20260810 11:33:59.593629 23072 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:33:59.610371 23072 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:33:59.616662 23072 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:33:59.617040 23072 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:33:59.641637 23072 rpc_server.cc:307] RPC server started. Bound to: 127.8.18.254:43799
I20260810 11:33:59.641656 23140 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.18.254:43799 every 8 connection(s)
I20260810 11:33:59.643602 23072 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
I20260810 11:33:59.646649 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 23072
I20260810 11:33:59.647795 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.18.193:41937
--local_ip_for_outbound_sockets=127.8.18.193
--tserver_master_addrs=127.8.18.254:43799
--webserver_port=43503
--webserver_interface=127.8.18.193
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.193.keytab
--principal=kudu/127.8.18.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.18.212:35253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 11:33:59.655578 23141 sys_catalog.cc:263] Verifying existing consensus state
I20260810 11:33:59.662395 23141 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 99954a220cf4425b894a741dd117548a: Bootstrap starting.
I20260810 11:33:59.687640 23141 log.cc:824] T 00000000000000000000000000000000 P 99954a220cf4425b894a741dd117548a: Log is configured to *not* fsync() on all Append() calls
I20260810 11:33:59.708139 23141 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 99954a220cf4425b894a741dd117548a: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 11:33:59.708945 23141 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 99954a220cf4425b894a741dd117548a: Bootstrap complete.
I20260810 11:33:59.720103 23141 raft_consensus.cc:348] T 00000000000000000000000000000000 P 99954a220cf4425b894a741dd117548a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99954a220cf4425b894a741dd117548a" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 43799 } }
I20260810 11:33:59.721084 23141 raft_consensus.cc:729] T 00000000000000000000000000000000 P 99954a220cf4425b894a741dd117548a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99954a220cf4425b894a741dd117548a, State: Initialized, Role: FOLLOWER
I20260810 11:33:59.721947 23141 consensus_queue.cc:260] T 00000000000000000000000000000000 P 99954a220cf4425b894a741dd117548a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99954a220cf4425b894a741dd117548a" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 43799 } }
I20260810 11:33:59.722232 23141 raft_consensus.cc:388] T 00000000000000000000000000000000 P 99954a220cf4425b894a741dd117548a [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 11:33:59.722363 23141 raft_consensus.cc:482] T 00000000000000000000000000000000 P 99954a220cf4425b894a741dd117548a [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 11:33:59.722571 23141 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 99954a220cf4425b894a741dd117548a [term 1 FOLLOWER]: Advancing to term 2
I20260810 11:33:59.727181 23141 raft_consensus.cc:504] T 00000000000000000000000000000000 P 99954a220cf4425b894a741dd117548a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99954a220cf4425b894a741dd117548a" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 43799 } }
I20260810 11:33:59.727872 23141 leader_election.cc:302] T 00000000000000000000000000000000 P 99954a220cf4425b894a741dd117548a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 99954a220cf4425b894a741dd117548a; no voters:
I20260810 11:33:59.728595 23141 leader_election.cc:288] T 00000000000000000000000000000000 P 99954a220cf4425b894a741dd117548a [CANDIDATE]: Term 2 election: Requested vote from peers
I20260810 11:33:59.728753 23145 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 99954a220cf4425b894a741dd117548a [term 2 FOLLOWER]: Leader election won for term 2
I20260810 11:33:59.730212 23145 raft_consensus.cc:686] T 00000000000000000000000000000000 P 99954a220cf4425b894a741dd117548a [term 2 LEADER]: Becoming Leader. State: Replica: 99954a220cf4425b894a741dd117548a, State: Running, Role: LEADER
I20260810 11:33:59.730983 23145 consensus_queue.cc:237] T 00000000000000000000000000000000 P 99954a220cf4425b894a741dd117548a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99954a220cf4425b894a741dd117548a" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 43799 } }
I20260810 11:33:59.731380 23141 sys_catalog.cc:564] T 00000000000000000000000000000000 P 99954a220cf4425b894a741dd117548a [sys.catalog]: configured and running, proceeding with master startup.
I20260810 11:33:59.733798 23147 sys_catalog.cc:455] T 00000000000000000000000000000000 P 99954a220cf4425b894a741dd117548a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 99954a220cf4425b894a741dd117548a. Latest consensus state: current_term: 2 leader_uuid: "99954a220cf4425b894a741dd117548a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99954a220cf4425b894a741dd117548a" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 43799 } } }
I20260810 11:33:59.734361 23147 sys_catalog.cc:458] T 00000000000000000000000000000000 P 99954a220cf4425b894a741dd117548a [sys.catalog]: This master's current role is: LEADER
I20260810 11:33:59.735020 23146 sys_catalog.cc:455] T 00000000000000000000000000000000 P 99954a220cf4425b894a741dd117548a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "99954a220cf4425b894a741dd117548a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99954a220cf4425b894a741dd117548a" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 43799 } } }
I20260810 11:33:59.737550 23146 sys_catalog.cc:458] T 00000000000000000000000000000000 P 99954a220cf4425b894a741dd117548a [sys.catalog]: This master's current role is: LEADER
I20260810 11:33:59.747292 23151 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 11:33:59.757432 23151 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 11:33:59.759778 23151 catalog_manager.cc:1259] Loaded cluster ID: b7dd071c324847619421b3caaec5f6e3
I20260810 11:33:59.760064 23151 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 11:33:59.765244 23151 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260810 11:33:59.777395 23151 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260810 11:33:59.785681 23151 catalog_manager.cc:1504] Loading token signing keys...
I20260810 11:33:59.787420 23151 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 99954a220cf4425b894a741dd117548a: Loaded TSK: 0
I20260810 11:33:59.788831 23151 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260810 11:34:00.107976 23143 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:34:00.108485 23143 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:34:00.108695 23143 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:34:00.122323 23143 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:34:00.122514 23143 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:34:00.122709 23143 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.18.193
I20260810 11:34:00.134301 23143 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.18.212:35253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.193.keytab
--principal=kudu/127.8.18.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.18.193:41937
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.18.193
--webserver_port=43503
--tserver_master_addrs=127.8.18.254:43799
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.18.193
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:34:00.135847 23143 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:34:00.137461 23143 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:34:00.155369 23169 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:34:00.156198 23168 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:34:00.157578 23171 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:34:00.157886 23143 server_base.cc:1034] running on GCE node
I20260810 11:34:00.158764 23143 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:34:00.160063 23143 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:34:00.161322 23143 hybrid_clock.cc:648] HybridClock initialized: now 1786361640161252 us; error 84 us; skew 500 ppm
I20260810 11:34:00.161751 23143 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:34:00 dist-test-slave-srgv krb5kdc[22509](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361640, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:34:00.165632 23143 init.cc:377] Logged in from keytab as kudu/127.8.18.193@KRBTEST.COM (short username kudu)
I20260810 11:34:00.167578 23143 webserver.cc:460] Webserver started at http://127.8.18.193:43503/ using document root <none> and password file <none>
I20260810 11:34:00.168402 23143 fs_manager.cc:359] Metadata directory not provided
I20260810 11:34:00.168577 23143 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:34:00.175351 23143 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260810 11:34:00.178961 23178 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:34:00.180753 23143 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260810 11:34:00.180980 23143 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/ts-0/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/ts-0/wal
uuid: "b53ea98f7b4b4a828f90fc4578efe84a"
format_stamp: "Formatted at 2026-08-10 11:33:56 on dist-test-slave-srgv"
server_key: "432487eb597c21bb93993d7fff75c65d"
server_key_iv: "fd74c204c2a17562810d2d473e2f02f4"
server_key_version: "encryptionkey@0"
I20260810 11:34:00.182173 23143 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:34:00.207584 23143 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:34:00.214759 23143 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:34:00.215289 23143 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:34:00.216758 23143 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:34:00.218565 23143 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:34:00.218740 23143 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:34:00.218907 23143 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:34:00.219022 23143 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:34:00.261809 23143 rpc_server.cc:307] RPC server started. Bound to: 127.8.18.193:41937
I20260810 11:34:00.261884 23293 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.18.193:41937 every 8 connection(s)
I20260810 11:34:00.263670 23143 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/ts-0/data/info.pb
I20260810 11:34:00.268198 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 23143
I20260810 11:34:00.269188 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.18.194:44985
--local_ip_for_outbound_sockets=127.8.18.194
--tserver_master_addrs=127.8.18.254:43799
--webserver_port=36647
--webserver_interface=127.8.18.194
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.194.keytab
--principal=kudu/127.8.18.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.18.212:35253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 11:34:00 dist-test-slave-srgv krb5kdc[22509](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361640, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.193@KRBTEST.COM for kudu/127.8.18.254@KRBTEST.COM
I20260810 11:34:00.299731 23297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:34:00.267838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:43799 (local address 127.8.18.193:58427)
0810 11:34:00.269220 (+ 1382us) negotiation.cc:107] Waiting for socket to connect
0810 11:34:00.269291 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0810 11:34:00.272728 (+ 3437us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:34:00.274946 (+ 2218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:34:00.274964 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:34:00.275928 (+ 964us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:34:00.276910 (+ 982us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:34:00.276947 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:34:00.279135 (+ 2188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:34:00.279145 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:34:00.281060 (+ 1915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:34:00.281084 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:34:00.283115 (+ 2031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:34:00.283122 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:34:00.283380 (+ 258us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:34:00.284365 (+ 985us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:34:00.284416 (+ 51us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:34:00.287392 (+ 2976us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:34:00.293543 (+ 6151us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:34:00.293561 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:34:00.293577 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:34:00.294208 (+ 631us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:34:00.294975 (+ 767us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:34:00.294980 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:34:00.294983 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:34:00.295374 (+ 391us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:34:00.296782 (+ 1408us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:34:00.296791 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:34:00.297254 (+ 463us) client_negotiation.cc:769] Sending connection context
0810 11:34:00.297655 (+ 401us) client_negotiation.cc:240] Negotiation successful
0810 11:34:00.298182 (+ 527us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":998,"thread_start_us":754,"threads_started":1}
I20260810 11:34:00.300834 23296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:34:00.267755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.193:58427 (local address 127.8.18.254:43799)
0810 11:34:00.268716 (+ 961us) server_negotiation.cc:206] Beginning negotiation
0810 11:34:00.268740 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0810 11:34:00.269503 (+ 763us) server_negotiation.cc:407] Connection header received
0810 11:34:00.273752 (+ 4249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:34:00.273769 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:34:00.274233 (+ 464us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:34:00.274606 (+ 373us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:34:00.277293 (+ 2687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:34:00.278832 (+ 1539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:34:00.281336 (+ 2504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:34:00.282794 (+ 1458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:34:00.283051 (+ 257us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:34:00.287865 (+ 4814us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:34:00.287890 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:34:00.287926 (+ 36us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:34:00.288003 (+ 77us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:34:00.293145 (+ 5142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:34:00.294427 (+ 1282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:34:00.294441 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:34:00.294452 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:34:00.294741 (+ 289us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:34:00.295621 (+ 880us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:34:00.295630 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:34:00.295636 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:34:00.296539 (+ 903us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:34:00.296784 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0810 11:34:00.297985 (+ 1201us) server_negotiation.cc:299] Negotiation successful
0810 11:34:00.298551 (+ 566us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":595,"thread_start_us":359,"threads_started":1}
I20260810 11:34:00.304638 23294 heartbeater.cc:347] Connected to a master server at 127.8.18.254:43799
I20260810 11:34:00.305042 23294 heartbeater.cc:464] Registering TS with master...
I20260810 11:34:00.306159 23294 heartbeater.cc:511] Master 127.8.18.254:43799 requested a full tablet report, sending...
I20260810 11:34:00.309089 23106 ts_manager.cc:194] Registered new tserver with Master: b53ea98f7b4b4a828f90fc4578efe84a (127.8.18.193:41937)
I20260810 11:34:00.311405 23106 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.18.193@KRBTEST.COM'} at 127.8.18.193:58427
W20260810 11:34:00.665535 23298 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:34:00.666009 23298 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:34:00.666205 23298 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:34:00.674316 23298 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:34:00.674499 23298 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:34:00.674703 23298 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.18.194
I20260810 11:34:00.686100 23298 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.18.212:35253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.194.keytab
--principal=kudu/127.8.18.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.18.194:44985
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.18.194
--webserver_port=36647
--tserver_master_addrs=127.8.18.254:43799
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.18.194
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:34:00.687631 23298 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:34:00.689105 23298 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:34:00.704440 23304 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:34:00.704511 23303 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:34:00.704540 23306 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:34:00.705829 23298 server_base.cc:1034] running on GCE node
I20260810 11:34:00.706665 23298 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:34:00.707934 23298 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:34:00.709240 23298 hybrid_clock.cc:648] HybridClock initialized: now 1786361640709197 us; error 43 us; skew 500 ppm
I20260810 11:34:00.709695 23298 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:34:00 dist-test-slave-srgv krb5kdc[22509](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361640, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:34:00.713387 23298 init.cc:377] Logged in from keytab as kudu/127.8.18.194@KRBTEST.COM (short username kudu)
I20260810 11:34:00.715416 23298 webserver.cc:460] Webserver started at http://127.8.18.194:36647/ using document root <none> and password file <none>
I20260810 11:34:00.716285 23298 fs_manager.cc:359] Metadata directory not provided
I20260810 11:34:00.716477 23298 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:34:00.722949 23298 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260810 11:34:00.726408 23313 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:34:00.728029 23298 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 11:34:00.728282 23298 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/ts-1/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/ts-1/wal
uuid: "abbaeefbabb941d8ae31fe6e637fdbc0"
format_stamp: "Formatted at 2026-08-10 11:33:57 on dist-test-slave-srgv"
server_key: "42d45e663bfadeb03076cdff4ecf7fef"
server_key_iv: "c89e0cbaa3e21eb63f117d46dd0343af"
server_key_version: "encryptionkey@0"
I20260810 11:34:00.729537 23298 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:34:00.751596 23298 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:34:00.756850 23298 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:34:00.757251 23298 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:34:00.758589 23298 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:34:00.760320 23298 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:34:00.760476 23298 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:34:00.760633 23298 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:34:00.760725 23298 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:34:00.798489 23298 rpc_server.cc:307] RPC server started. Bound to: 127.8.18.194:44985
I20260810 11:34:00.798503 23428 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.18.194:44985 every 8 connection(s)
I20260810 11:34:00.800530 23298 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/ts-1/data/info.pb
I20260810 11:34:00.807504 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 23298
I20260810 11:34:00.808261 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.18.195:43839
--local_ip_for_outbound_sockets=127.8.18.195
--tserver_master_addrs=127.8.18.254:43799
--webserver_port=35623
--webserver_interface=127.8.18.195
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.195.keytab
--principal=kudu/127.8.18.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.18.212:35253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 11:34:00 dist-test-slave-srgv krb5kdc[22509](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361640, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.194@KRBTEST.COM for kudu/127.8.18.254@KRBTEST.COM
I20260810 11:34:00.824494 23431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:34:00.803968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.194:43897 (local address 127.8.18.254:43799)
0810 11:34:00.804574 (+ 606us) server_negotiation.cc:206] Beginning negotiation
0810 11:34:00.804579 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:34:00.805666 (+ 1087us) server_negotiation.cc:407] Connection header received
0810 11:34:00.807863 (+ 2197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:34:00.807869 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:34:00.807976 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:34:00.808106 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:34:00.810053 (+ 1947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:34:00.810800 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:34:00.812505 (+ 1705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:34:00.812986 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:34:00.813147 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:34:00.817238 (+ 4091us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:34:00.817245 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:34:00.817248 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:34:00.817299 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:34:00.820332 (+ 3033us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:34:00.821460 (+ 1128us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:34:00.821469 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:34:00.821473 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:34:00.821711 (+ 238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:34:00.822394 (+ 683us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:34:00.822400 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:34:00.822403 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:34:00.822733 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:34:00.822906 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0810 11:34:00.823764 (+ 858us) server_negotiation.cc:299] Negotiation successful
0810 11:34:00.824062 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":499,"thread_start_us":348,"threads_started":1}
I20260810 11:34:00.825706 23432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:34:00.804533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:43799 (local address 127.8.18.194:43897)
0810 11:34:00.805396 (+ 863us) negotiation.cc:107] Waiting for socket to connect
0810 11:34:00.805521 (+ 125us) client_negotiation.cc:174] Beginning negotiation
0810 11:34:00.807482 (+ 1961us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:34:00.808454 (+ 972us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:34:00.808468 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:34:00.809093 (+ 625us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:34:00.809832 (+ 739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:34:00.809854 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:34:00.811006 (+ 1152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:34:00.811010 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:34:00.812324 (+ 1314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:34:00.812339 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:34:00.813286 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:34:00.813295 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:34:00.813648 (+ 353us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:34:00.814461 (+ 813us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:34:00.814492 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:34:00.816913 (+ 2421us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:34:00.820544 (+ 3631us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:34:00.820555 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:34:00.820583 (+ 28us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:34:00.821233 (+ 650us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:34:00.821869 (+ 636us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:34:00.821878 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:34:00.821882 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:34:00.822246 (+ 364us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:34:00.822882 (+ 636us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:34:00.822891 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:34:00.823346 (+ 455us) client_negotiation.cc:769] Sending connection context
0810 11:34:00.823665 (+ 319us) client_negotiation.cc:240] Negotiation successful
0810 11:34:00.824302 (+ 637us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":568,"thread_start_us":345,"threads_started":1}
I20260810 11:34:00.827553 23429 heartbeater.cc:347] Connected to a master server at 127.8.18.254:43799
I20260810 11:34:00.827924 23429 heartbeater.cc:464] Registering TS with master...
I20260810 11:34:00.828747 23429 heartbeater.cc:511] Master 127.8.18.254:43799 requested a full tablet report, sending...
I20260810 11:34:00.830696 23106 ts_manager.cc:194] Registered new tserver with Master: abbaeefbabb941d8ae31fe6e637fdbc0 (127.8.18.194:44985)
I20260810 11:34:00.832150 23106 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.18.194@KRBTEST.COM'} at 127.8.18.194:43897
W20260810 11:34:01.182024 23433 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:34:01.182353 23433 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:34:01.182463 23433 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:34:01.191198 23433 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:34:01.191346 23433 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:34:01.191468 23433 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.18.195
I20260810 11:34:01.202335 23433 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.18.212:35253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.195.keytab
--principal=kudu/127.8.18.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.18.195:43839
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.18.195
--webserver_port=35623
--tserver_master_addrs=127.8.18.254:43799
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.18.195
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:34:01.203682 23433 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:34:01.205121 23433 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:34:01.220405 23442 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:34:01.220386 23440 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:34:01.220772 23438 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:34:01.221525 23433 server_base.cc:1034] running on GCE node
I20260810 11:34:01.222297 23433 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:34:01.223407 23433 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:34:01.224682 23433 hybrid_clock.cc:648] HybridClock initialized: now 1786361641224621 us; error 54 us; skew 500 ppm
I20260810 11:34:01.225138 23433 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:34:01 dist-test-slave-srgv krb5kdc[22509](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361641, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:34:01.228948 23433 init.cc:377] Logged in from keytab as kudu/127.8.18.195@KRBTEST.COM (short username kudu)
I20260810 11:34:01.230679 23433 webserver.cc:460] Webserver started at http://127.8.18.195:35623/ using document root <none> and password file <none>
I20260810 11:34:01.231534 23433 fs_manager.cc:359] Metadata directory not provided
I20260810 11:34:01.231668 23433 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:34:01.238407 23433 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260810 11:34:01.241681 23449 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:34:01.243606 23433 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260810 11:34:01.243875 23433 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/ts-2/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/ts-2/wal
uuid: "52042505dd7f42e990ee5684ada73014"
format_stamp: "Formatted at 2026-08-10 11:33:58 on dist-test-slave-srgv"
server_key: "5de592d87e7c1bb273f31396872e2cd2"
server_key_iv: "50211b8d617be8277b70ee6472c7f469"
server_key_version: "encryptionkey@0"
I20260810 11:34:01.245298 23433 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:34:01.278875 23433 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:34:01.284600 23433 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:34:01.284886 23433 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:34:01.286026 23433 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:34:01.287912 23433 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:34:01.288064 23433 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:34:01.288228 23433 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:34:01.288334 23433 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:34:01.315850 23294 heartbeater.cc:503] Master 127.8.18.254:43799 was elected leader, sending a full tablet report...
I20260810 11:34:01.326923 23433 rpc_server.cc:307] RPC server started. Bound to: 127.8.18.195:43839
I20260810 11:34:01.327160 23562 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.18.195:43839 every 8 connection(s)
I20260810 11:34:01.328555 23433 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/ts-2/data/info.pb
I20260810 11:34:01.331616 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 23433
Aug 10 11:34:01 dist-test-slave-srgv krb5kdc[22509](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361641, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.195@KRBTEST.COM for kudu/127.8.18.254@KRBTEST.COM
I20260810 11:34:01.357040 23567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:34:01.332651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48900 (local address 127.8.18.254:43799)
0810 11:34:01.338140 (+ 5489us) server_negotiation.cc:206] Beginning negotiation
0810 11:34:01.338150 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 11:34:01.338839 (+ 689us) server_negotiation.cc:407] Connection header received
0810 11:34:01.338949 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:34:01.338956 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:34:01.339042 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:34:01.339178 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:34:01.340857 (+ 1679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:34:01.341962 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:34:01.344250 (+ 2288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:34:01.345256 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:34:01.345519 (+ 263us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:34:01.349676 (+ 4157us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:34:01.349687 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:34:01.349692 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:34:01.349752 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:34:01.353785 (+ 4033us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:34:01.354682 (+ 897us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:34:01.354691 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:34:01.354700 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:34:01.354866 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:34:01.355416 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:34:01.355421 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:34:01.355423 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:34:01.355782 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:34:01.356076 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0810 11:34:01.356405 (+ 329us) server_negotiation.cc:299] Negotiation successful
0810 11:34:01.356601 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":197,"server-negotiator.queue_time_us":5271,"spinlock_wait_cycles":13056,"thread_start_us":5032,"threads_started":1}
I20260810 11:34:01.358939 23565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:34:01.331703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.195:41659 (local address 127.8.18.254:43799)
0810 11:34:01.332362 (+ 659us) server_negotiation.cc:206] Beginning negotiation
0810 11:34:01.332367 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:34:01.333805 (+ 1438us) server_negotiation.cc:407] Connection header received
0810 11:34:01.337806 (+ 4001us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:34:01.337813 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:34:01.337941 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:34:01.338014 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:34:01.340338 (+ 2324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:34:01.341514 (+ 1176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:34:01.344250 (+ 2736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:34:01.345256 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:34:01.345513 (+ 257us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:34:01.351419 (+ 5906us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:34:01.351426 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:34:01.351431 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:34:01.351481 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:34:01.355050 (+ 3569us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:34:01.356001 (+ 951us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:34:01.356006 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:34:01.356010 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:34:01.356161 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:34:01.356788 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:34:01.356794 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:34:01.356799 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:34:01.357159 (+ 360us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:34:01.357363 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0810 11:34:01.358246 (+ 883us) server_negotiation.cc:299] Negotiation successful
0810 11:34:01.358471 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":139,"server-negotiator.queue_time_us":522,"spinlock_wait_cycles":896,"thread_start_us":390,"threads_started":1}
I20260810 11:34:01.360157 23566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:34:01.332343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:43799 (local address 127.8.18.195:41659)
0810 11:34:01.333460 (+ 1117us) negotiation.cc:107] Waiting for socket to connect
0810 11:34:01.333545 (+ 85us) client_negotiation.cc:174] Beginning negotiation
0810 11:34:01.337234 (+ 3689us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:34:01.338553 (+ 1319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:34:01.338563 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:34:01.339481 (+ 918us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:34:01.340131 (+ 650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:34:01.340153 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:34:01.341708 (+ 1555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:34:01.341713 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:34:01.343991 (+ 2278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:34:01.344040 (+ 49us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:34:01.345705 (+ 1665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:34:01.345716 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:34:01.346096 (+ 380us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:34:01.346869 (+ 773us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:34:01.346905 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:34:01.351086 (+ 4181us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:34:01.355250 (+ 4164us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:34:01.355262 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:34:01.355282 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:34:01.355821 (+ 539us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:34:01.356338 (+ 517us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:34:01.356343 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:34:01.356347 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:34:01.356649 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:34:01.357342 (+ 693us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:34:01.357351 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:34:01.357833 (+ 482us) client_negotiation.cc:769] Sending connection context
0810 11:34:01.358163 (+ 330us) client_negotiation.cc:240] Negotiation successful
0810 11:34:01.358795 (+ 632us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":665,"thread_start_us":392,"threads_started":1}
I20260810 11:34:01.361806 23563 heartbeater.cc:347] Connected to a master server at 127.8.18.254:43799
I20260810 11:34:01.362172 23563 heartbeater.cc:464] Registering TS with master...
I20260810 11:34:01.363020 23563 heartbeater.cc:511] Master 127.8.18.254:43799 requested a full tablet report, sending...
I20260810 11:34:01.364957 23106 ts_manager.cc:194] Registered new tserver with Master: 52042505dd7f42e990ee5684ada73014 (127.8.18.195:43839)
I20260810 11:34:01.365919 23106 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.18.195@KRBTEST.COM'} at 127.8.18.195:41659
I20260810 11:34:01.373886 8267 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 11:34:01.382651 23565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:34:01.375477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48906 (local address 127.8.18.254:43799)
0810 11:34:01.375718 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0810 11:34:01.375724 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:34:01.376306 (+ 582us) server_negotiation.cc:407] Connection header received
0810 11:34:01.376536 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:34:01.376552 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:34:01.376747 (+ 195us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:34:01.376915 (+ 168us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 11:34:01.377525 (+ 610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:34:01.378295 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:34:01.379976 (+ 1681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:34:01.380482 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:34:01.380603 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:34:01.381300 (+ 697us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 11:34:01.382018 (+ 718us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 11:34:01.382152 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0810 11:34:01.382313 (+ 161us) server_negotiation.cc:299] Negotiation successful
0810 11:34:01.382400 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106}
I20260810 11:34:01.383762 23106 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:48906:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 11:34:01.391667 23106 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 11:34:01.420159 23578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:34:01.406684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45656 (local address 127.8.18.193:41937)
0810 11:34:01.411294 (+ 4610us) server_negotiation.cc:206] Beginning negotiation
0810 11:34:01.411305 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 11:34:01.411342 (+ 37us) server_negotiation.cc:407] Connection header received
0810 11:34:01.411454 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:34:01.411465 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:34:01.411783 (+ 318us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:34:01.411999 (+ 216us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:34:01.413133 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:34:01.414908 (+ 1775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:34:01.416863 (+ 1955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:34:01.418709 (+ 1846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:34:01.418869 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:34:01.419135 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0810 11:34:01.419554 (+ 419us) server_negotiation.cc:299] Negotiation successful
0810 11:34:01.419773 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4395,"thread_start_us":4201,"threads_started":1}
I20260810 11:34:01.420686 23579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:34:01.407003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43470 (local address 127.8.18.194:44985)
0810 11:34:01.409988 (+ 2985us) server_negotiation.cc:206] Beginning negotiation
0810 11:34:01.410000 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 11:34:01.410041 (+ 41us) server_negotiation.cc:407] Connection header received
0810 11:34:01.410181 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:34:01.410190 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:34:01.410528 (+ 338us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:34:01.411632 (+ 1104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:34:01.411945 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:34:01.413754 (+ 1809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:34:01.417562 (+ 3808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:34:01.419241 (+ 1679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:34:01.419430 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:34:01.419637 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0810 11:34:01.419957 (+ 320us) server_negotiation.cc:299] Negotiation successful
0810 11:34:01.420136 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2813,"thread_start_us":2688,"threads_started":1}
I20260810 11:34:01.420864 23576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:34:01.406290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.194:44985 (local address 127.0.0.1:43470)
0810 11:34:01.407259 (+ 969us) negotiation.cc:107] Waiting for socket to connect
0810 11:34:01.407305 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0810 11:34:01.407523 (+ 218us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:34:01.410843 (+ 3320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:34:01.410854 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:34:01.410897 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:34:01.411369 (+ 472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:34:01.411386 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:34:01.413933 (+ 2547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:34:01.413940 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:34:01.416961 (+ 3021us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:34:01.416979 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:34:01.419518 (+ 2539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:34:01.419524 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:34:01.419725 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:34:01.419756 (+ 31us) client_negotiation.cc:769] Sending connection context
0810 11:34:01.419864 (+ 108us) client_negotiation.cc:240] Negotiation successful
0810 11:34:01.420107 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":636,"mutex_wait_us":1,"thread_start_us":915,"threads_started":1}
I20260810 11:34:01.421211 23577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:34:01.406338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.193:41937 (local address 127.0.0.1:45656)
0810 11:34:01.410430 (+ 4092us) negotiation.cc:107] Waiting for socket to connect
0810 11:34:01.410457 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0810 11:34:01.410609 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:34:01.412181 (+ 1572us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:34:01.412186 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:34:01.412224 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:34:01.412773 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:34:01.412790 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:34:01.415034 (+ 2244us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:34:01.415042 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:34:01.416645 (+ 1603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:34:01.416662 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:34:01.418988 (+ 2326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:34:01.418995 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:34:01.419230 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:34:01.419293 (+ 63us) client_negotiation.cc:769] Sending connection context
0810 11:34:01.420504 (+ 1211us) client_negotiation.cc:240] Negotiation successful
0810 11:34:01.420675 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3950,"thread_start_us":3815,"threads_started":1}
I20260810 11:34:01.423398 23581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:34:01.412130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.195:43839 (local address 127.0.0.1:34824)
0810 11:34:01.412956 (+ 826us) negotiation.cc:107] Waiting for socket to connect
0810 11:34:01.412979 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0810 11:34:01.413123 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:34:01.414256 (+ 1133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:34:01.414262 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:34:01.414291 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:34:01.414849 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:34:01.414864 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:34:01.417036 (+ 2172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:34:01.417043 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:34:01.419503 (+ 2460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:34:01.419516 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:34:01.421856 (+ 2340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:34:01.422051 (+ 195us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:34:01.422273 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:34:01.422309 (+ 36us) client_negotiation.cc:769] Sending connection context
0810 11:34:01.422546 (+ 237us) client_negotiation.cc:240] Negotiation successful
0810 11:34:01.422711 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":612,"thread_start_us":403,"threads_started":1}
I20260810 11:34:01.423825 23580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:34:01.411279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34824 (local address 127.8.18.195:43839)
0810 11:34:01.412072 (+ 793us) server_negotiation.cc:206] Beginning negotiation
0810 11:34:01.412081 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 11:34:01.413256 (+ 1175us) server_negotiation.cc:407] Connection header received
0810 11:34:01.413373 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:34:01.413381 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:34:01.413653 (+ 272us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:34:01.413889 (+ 236us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:34:01.415103 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:34:01.416869 (+ 1766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:34:01.419741 (+ 2872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:34:01.421533 (+ 1792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:34:01.421673 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:34:01.421918 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0810 11:34:01.423218 (+ 1300us) server_negotiation.cc:299] Negotiation successful
0810 11:34:01.423437 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":633,"thread_start_us":467,"threads_started":1}
I20260810 11:34:01.425727 23363 tablet_service.cc:1467] Processing CreateTablet for tablet 55d2acb48bd04b95913aa61fbc158fca (DEFAULT_TABLE table=test-table [id=8016185a0d6d47f19894516eb0977b9c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:34:01.425750 23227 tablet_service.cc:1467] Processing CreateTablet for tablet 55d2acb48bd04b95913aa61fbc158fca (DEFAULT_TABLE table=test-table [id=8016185a0d6d47f19894516eb0977b9c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:34:01.427323 23227 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 55d2acb48bd04b95913aa61fbc158fca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:34:01.427323 23363 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 55d2acb48bd04b95913aa61fbc158fca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:34:01.428151 23497 tablet_service.cc:1467] Processing CreateTablet for tablet 55d2acb48bd04b95913aa61fbc158fca (DEFAULT_TABLE table=test-table [id=8016185a0d6d47f19894516eb0977b9c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:34:01.429883 23497 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 55d2acb48bd04b95913aa61fbc158fca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:34:01.443126 23582 tablet_bootstrap.cc:492] T 55d2acb48bd04b95913aa61fbc158fca P abbaeefbabb941d8ae31fe6e637fdbc0: Bootstrap starting.
I20260810 11:34:01.445943 23583 tablet_bootstrap.cc:492] T 55d2acb48bd04b95913aa61fbc158fca P b53ea98f7b4b4a828f90fc4578efe84a: Bootstrap starting.
I20260810 11:34:01.448865 23584 tablet_bootstrap.cc:492] T 55d2acb48bd04b95913aa61fbc158fca P 52042505dd7f42e990ee5684ada73014: Bootstrap starting.
I20260810 11:34:01.449348 23582 tablet_bootstrap.cc:654] T 55d2acb48bd04b95913aa61fbc158fca P abbaeefbabb941d8ae31fe6e637fdbc0: Neither blocks nor log segments found. Creating new log.
I20260810 11:34:01.450201 23583 tablet_bootstrap.cc:654] T 55d2acb48bd04b95913aa61fbc158fca P b53ea98f7b4b4a828f90fc4578efe84a: Neither blocks nor log segments found. Creating new log.
I20260810 11:34:01.451063 23582 log.cc:824] T 55d2acb48bd04b95913aa61fbc158fca P abbaeefbabb941d8ae31fe6e637fdbc0: Log is configured to *not* fsync() on all Append() calls
I20260810 11:34:01.451570 23583 log.cc:824] T 55d2acb48bd04b95913aa61fbc158fca P b53ea98f7b4b4a828f90fc4578efe84a: Log is configured to *not* fsync() on all Append() calls
I20260810 11:34:01.452677 23584 tablet_bootstrap.cc:654] T 55d2acb48bd04b95913aa61fbc158fca P 52042505dd7f42e990ee5684ada73014: Neither blocks nor log segments found. Creating new log.
I20260810 11:34:01.453923 23584 log.cc:824] T 55d2acb48bd04b95913aa61fbc158fca P 52042505dd7f42e990ee5684ada73014: Log is configured to *not* fsync() on all Append() calls
I20260810 11:34:01.455112 23582 tablet_bootstrap.cc:492] T 55d2acb48bd04b95913aa61fbc158fca P abbaeefbabb941d8ae31fe6e637fdbc0: No bootstrap required, opened a new log
I20260810 11:34:01.455479 23582 ts_tablet_manager.cc:1397] T 55d2acb48bd04b95913aa61fbc158fca P abbaeefbabb941d8ae31fe6e637fdbc0: Time spent bootstrapping tablet: real 0.013s user 0.004s sys 0.004s
I20260810 11:34:01.455889 23583 tablet_bootstrap.cc:492] T 55d2acb48bd04b95913aa61fbc158fca P b53ea98f7b4b4a828f90fc4578efe84a: No bootstrap required, opened a new log
I20260810 11:34:01.456595 23583 ts_tablet_manager.cc:1397] T 55d2acb48bd04b95913aa61fbc158fca P b53ea98f7b4b4a828f90fc4578efe84a: Time spent bootstrapping tablet: real 0.011s user 0.009s sys 0.000s
I20260810 11:34:01.457751 23584 tablet_bootstrap.cc:492] T 55d2acb48bd04b95913aa61fbc158fca P 52042505dd7f42e990ee5684ada73014: No bootstrap required, opened a new log
I20260810 11:34:01.458030 23584 ts_tablet_manager.cc:1397] T 55d2acb48bd04b95913aa61fbc158fca P 52042505dd7f42e990ee5684ada73014: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260810 11:34:01.464924 23583 raft_consensus.cc:348] T 55d2acb48bd04b95913aa61fbc158fca P b53ea98f7b4b4a828f90fc4578efe84a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b53ea98f7b4b4a828f90fc4578efe84a" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 41937 } } peers { permanent_uuid: "abbaeefbabb941d8ae31fe6e637fdbc0" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 44985 } } peers { permanent_uuid: "52042505dd7f42e990ee5684ada73014" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 43839 } }
I20260810 11:34:01.465509 23583 raft_consensus.cc:374] T 55d2acb48bd04b95913aa61fbc158fca P b53ea98f7b4b4a828f90fc4578efe84a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:34:01.465622 23583 raft_consensus.cc:729] T 55d2acb48bd04b95913aa61fbc158fca P b53ea98f7b4b4a828f90fc4578efe84a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b53ea98f7b4b4a828f90fc4578efe84a, State: Initialized, Role: FOLLOWER
I20260810 11:34:01.466401 23583 consensus_queue.cc:260] T 55d2acb48bd04b95913aa61fbc158fca P b53ea98f7b4b4a828f90fc4578efe84a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b53ea98f7b4b4a828f90fc4578efe84a" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 41937 } } peers { permanent_uuid: "abbaeefbabb941d8ae31fe6e637fdbc0" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 44985 } } peers { permanent_uuid: "52042505dd7f42e990ee5684ada73014" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 43839 } }
I20260810 11:34:01.467825 23582 raft_consensus.cc:348] T 55d2acb48bd04b95913aa61fbc158fca P abbaeefbabb941d8ae31fe6e637fdbc0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b53ea98f7b4b4a828f90fc4578efe84a" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 41937 } } peers { permanent_uuid: "abbaeefbabb941d8ae31fe6e637fdbc0" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 44985 } } peers { permanent_uuid: "52042505dd7f42e990ee5684ada73014" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 43839 } }
I20260810 11:34:01.468431 23582 raft_consensus.cc:374] T 55d2acb48bd04b95913aa61fbc158fca P abbaeefbabb941d8ae31fe6e637fdbc0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:34:01.468554 23582 raft_consensus.cc:729] T 55d2acb48bd04b95913aa61fbc158fca P abbaeefbabb941d8ae31fe6e637fdbc0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abbaeefbabb941d8ae31fe6e637fdbc0, State: Initialized, Role: FOLLOWER
I20260810 11:34:01.469563 23582 consensus_queue.cc:260] T 55d2acb48bd04b95913aa61fbc158fca P abbaeefbabb941d8ae31fe6e637fdbc0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b53ea98f7b4b4a828f90fc4578efe84a" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 41937 } } peers { permanent_uuid: "abbaeefbabb941d8ae31fe6e637fdbc0" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 44985 } } peers { permanent_uuid: "52042505dd7f42e990ee5684ada73014" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 43839 } }
I20260810 11:34:01.470511 23584 raft_consensus.cc:348] T 55d2acb48bd04b95913aa61fbc158fca P 52042505dd7f42e990ee5684ada73014 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b53ea98f7b4b4a828f90fc4578efe84a" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 41937 } } peers { permanent_uuid: "abbaeefbabb941d8ae31fe6e637fdbc0" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 44985 } } peers { permanent_uuid: "52042505dd7f42e990ee5684ada73014" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 43839 } }
I20260810 11:34:01.471009 23583 ts_tablet_manager.cc:1428] T 55d2acb48bd04b95913aa61fbc158fca P b53ea98f7b4b4a828f90fc4578efe84a: Time spent starting tablet: real 0.014s user 0.006s sys 0.008s
I20260810 11:34:01.471211 23584 raft_consensus.cc:374] T 55d2acb48bd04b95913aa61fbc158fca P 52042505dd7f42e990ee5684ada73014 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:34:01.471342 23584 raft_consensus.cc:729] T 55d2acb48bd04b95913aa61fbc158fca P 52042505dd7f42e990ee5684ada73014 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52042505dd7f42e990ee5684ada73014, State: Initialized, Role: FOLLOWER
I20260810 11:34:01.473694 23582 ts_tablet_manager.cc:1428] T 55d2acb48bd04b95913aa61fbc158fca P abbaeefbabb941d8ae31fe6e637fdbc0: Time spent starting tablet: real 0.018s user 0.020s sys 0.000s
I20260810 11:34:01.473721 23584 consensus_queue.cc:260] T 55d2acb48bd04b95913aa61fbc158fca P 52042505dd7f42e990ee5684ada73014 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b53ea98f7b4b4a828f90fc4578efe84a" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 41937 } } peers { permanent_uuid: "abbaeefbabb941d8ae31fe6e637fdbc0" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 44985 } } peers { permanent_uuid: "52042505dd7f42e990ee5684ada73014" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 43839 } }
I20260810 11:34:01.474439 23429 heartbeater.cc:503] Master 127.8.18.254:43799 was elected leader, sending a full tablet report...
I20260810 11:34:01.475353 23563 heartbeater.cc:503] Master 127.8.18.254:43799 was elected leader, sending a full tablet report...
I20260810 11:34:01.476276 23584 ts_tablet_manager.cc:1428] T 55d2acb48bd04b95913aa61fbc158fca P 52042505dd7f42e990ee5684ada73014: Time spent starting tablet: real 0.018s user 0.016s sys 0.000s
I20260810 11:34:01.479768 23588 raft_consensus.cc:482] T 55d2acb48bd04b95913aa61fbc158fca P b53ea98f7b4b4a828f90fc4578efe84a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 11:34:01.480110 23588 raft_consensus.cc:504] T 55d2acb48bd04b95913aa61fbc158fca P b53ea98f7b4b4a828f90fc4578efe84a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b53ea98f7b4b4a828f90fc4578efe84a" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 41937 } } peers { permanent_uuid: "abbaeefbabb941d8ae31fe6e637fdbc0" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 44985 } } peers { permanent_uuid: "52042505dd7f42e990ee5684ada73014" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 43839 } }
I20260810 11:34:01.482535 23588 leader_election.cc:288] T 55d2acb48bd04b95913aa61fbc158fca P b53ea98f7b4b4a828f90fc4578efe84a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers abbaeefbabb941d8ae31fe6e637fdbc0 (127.8.18.194:44985), 52042505dd7f42e990ee5684ada73014 (127.8.18.195:43839)
I20260810 11:34:01.490648 23579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:34:01.482882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.193:35495 (local address 127.8.18.194:44985)
0810 11:34:01.483092 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0810 11:34:01.483101 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 11:34:01.483865 (+ 764us) server_negotiation.cc:407] Connection header received
0810 11:34:01.484106 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:34:01.484113 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:34:01.484203 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:34:01.484364 (+ 161us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:34:01.485098 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:34:01.486129 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:34:01.488436 (+ 2307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:34:01.489499 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:34:01.489635 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:34:01.489716 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0810 11:34:01.490161 (+ 445us) server_negotiation.cc:299] Negotiation successful
0810 11:34:01.490368 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260810 11:34:01.490654 23591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:34:01.482868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.194:44985 (local address 127.8.18.193:35495)
0810 11:34:01.483699 (+ 831us) negotiation.cc:107] Waiting for socket to connect
0810 11:34:01.483721 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 11:34:01.483916 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:34:01.484385 (+ 469us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:34:01.484389 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:34:01.484457 (+ 68us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:34:01.484916 (+ 459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:34:01.484934 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:34:01.486293 (+ 1359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:34:01.486299 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:34:01.488245 (+ 1946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:34:01.488260 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:34:01.489729 (+ 1469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:34:01.489737 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:34:01.489931 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:34:01.489971 (+ 40us) client_negotiation.cc:769] Sending connection context
0810 11:34:01.490086 (+ 115us) client_negotiation.cc:240] Negotiation successful
0810 11:34:01.490254 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":633,"thread_start_us":391,"threads_started":1}
I20260810 11:34:01.491010 23580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:34:01.482979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.193:46755 (local address 127.8.18.195:43839)
0810 11:34:01.483204 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0810 11:34:01.483213 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 11:34:01.484457 (+ 1244us) server_negotiation.cc:407] Connection header received
0810 11:34:01.484709 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:34:01.484714 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:34:01.484854 (+ 140us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:34:01.484982 (+ 128us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:34:01.485694 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:34:01.486714 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:34:01.488873 (+ 2159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:34:01.489997 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:34:01.490169 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:34:01.490241 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0810 11:34:01.490634 (+ 393us) server_negotiation.cc:299] Negotiation successful
0810 11:34:01.490789 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260810 11:34:01.491597 23592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:34:01.483490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.195:43839 (local address 127.8.18.193:46755)
0810 11:34:01.484331 (+ 841us) negotiation.cc:107] Waiting for socket to connect
0810 11:34:01.484353 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 11:34:01.484545 (+ 192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:34:01.485027 (+ 482us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:34:01.485032 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:34:01.485061 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:34:01.485473 (+ 412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:34:01.485488 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:34:01.486882 (+ 1394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:34:01.486887 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:34:01.488680 (+ 1793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:34:01.488696 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:34:01.490253 (+ 1557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:34:01.490260 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:34:01.490440 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:34:01.490474 (+ 34us) client_negotiation.cc:769] Sending connection context
0810 11:34:01.490575 (+ 101us) client_negotiation.cc:240] Negotiation successful
0810 11:34:01.490723 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":706,"mutex_wait_us":29,"spinlock_wait_cycles":29568,"thread_start_us":442,"threads_started":1}
I20260810 11:34:01.491919 23383 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "55d2acb48bd04b95913aa61fbc158fca" candidate_uuid: "b53ea98f7b4b4a828f90fc4578efe84a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abbaeefbabb941d8ae31fe6e637fdbc0" is_pre_election: true
I20260810 11:34:01.492547 23383 raft_consensus.cc:2454] T 55d2acb48bd04b95913aa61fbc158fca P abbaeefbabb941d8ae31fe6e637fdbc0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b53ea98f7b4b4a828f90fc4578efe84a in term 0.
I20260810 11:34:01.492722 23517 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "55d2acb48bd04b95913aa61fbc158fca" candidate_uuid: "b53ea98f7b4b4a828f90fc4578efe84a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52042505dd7f42e990ee5684ada73014" is_pre_election: true
I20260810 11:34:01.493242 23517 raft_consensus.cc:2454] T 55d2acb48bd04b95913aa61fbc158fca P 52042505dd7f42e990ee5684ada73014 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b53ea98f7b4b4a828f90fc4578efe84a in term 0.
I20260810 11:34:01.493307 23183 leader_election.cc:302] T 55d2acb48bd04b95913aa61fbc158fca P b53ea98f7b4b4a828f90fc4578efe84a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: abbaeefbabb941d8ae31fe6e637fdbc0, b53ea98f7b4b4a828f90fc4578efe84a; no voters:
I20260810 11:34:01.493852 23588 raft_consensus.cc:2781] T 55d2acb48bd04b95913aa61fbc158fca P b53ea98f7b4b4a828f90fc4578efe84a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 11:34:01.493981 23588 raft_consensus.cc:482] T 55d2acb48bd04b95913aa61fbc158fca P b53ea98f7b4b4a828f90fc4578efe84a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 11:34:01.494124 23588 raft_consensus.cc:3037] T 55d2acb48bd04b95913aa61fbc158fca P b53ea98f7b4b4a828f90fc4578efe84a [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:34:01.496500 23588 raft_consensus.cc:504] T 55d2acb48bd04b95913aa61fbc158fca P b53ea98f7b4b4a828f90fc4578efe84a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b53ea98f7b4b4a828f90fc4578efe84a" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 41937 } } peers { permanent_uuid: "abbaeefbabb941d8ae31fe6e637fdbc0" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 44985 } } peers { permanent_uuid: "52042505dd7f42e990ee5684ada73014" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 43839 } }
I20260810 11:34:01.497248 23588 leader_election.cc:288] T 55d2acb48bd04b95913aa61fbc158fca P b53ea98f7b4b4a828f90fc4578efe84a [CANDIDATE]: Term 1 election: Requested vote from peers abbaeefbabb941d8ae31fe6e637fdbc0 (127.8.18.194:44985), 52042505dd7f42e990ee5684ada73014 (127.8.18.195:43839)
I20260810 11:34:01.497871 23517 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "55d2acb48bd04b95913aa61fbc158fca" candidate_uuid: "b53ea98f7b4b4a828f90fc4578efe84a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52042505dd7f42e990ee5684ada73014"
I20260810 11:34:01.497912 23383 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "55d2acb48bd04b95913aa61fbc158fca" candidate_uuid: "b53ea98f7b4b4a828f90fc4578efe84a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abbaeefbabb941d8ae31fe6e637fdbc0"
I20260810 11:34:01.498103 23517 raft_consensus.cc:3037] T 55d2acb48bd04b95913aa61fbc158fca P 52042505dd7f42e990ee5684ada73014 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:34:01.498160 23383 raft_consensus.cc:3037] T 55d2acb48bd04b95913aa61fbc158fca P abbaeefbabb941d8ae31fe6e637fdbc0 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:34:01.500900 23383 raft_consensus.cc:2454] T 55d2acb48bd04b95913aa61fbc158fca P abbaeefbabb941d8ae31fe6e637fdbc0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b53ea98f7b4b4a828f90fc4578efe84a in term 1.
I20260810 11:34:01.500921 23517 raft_consensus.cc:2454] T 55d2acb48bd04b95913aa61fbc158fca P 52042505dd7f42e990ee5684ada73014 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b53ea98f7b4b4a828f90fc4578efe84a in term 1.
I20260810 11:34:01.501518 23183 leader_election.cc:302] T 55d2acb48bd04b95913aa61fbc158fca P b53ea98f7b4b4a828f90fc4578efe84a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: abbaeefbabb941d8ae31fe6e637fdbc0, b53ea98f7b4b4a828f90fc4578efe84a; no voters:
I20260810 11:34:01.501914 23588 raft_consensus.cc:2781] T 55d2acb48bd04b95913aa61fbc158fca P b53ea98f7b4b4a828f90fc4578efe84a [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:34:01.502521 23588 raft_consensus.cc:686] T 55d2acb48bd04b95913aa61fbc158fca P b53ea98f7b4b4a828f90fc4578efe84a [term 1 LEADER]: Becoming Leader. State: Replica: b53ea98f7b4b4a828f90fc4578efe84a, State: Running, Role: LEADER
I20260810 11:34:01.503156 23588 consensus_queue.cc:237] T 55d2acb48bd04b95913aa61fbc158fca P b53ea98f7b4b4a828f90fc4578efe84a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b53ea98f7b4b4a828f90fc4578efe84a" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 41937 } } peers { permanent_uuid: "abbaeefbabb941d8ae31fe6e637fdbc0" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 44985 } } peers { permanent_uuid: "52042505dd7f42e990ee5684ada73014" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 43839 } }
I20260810 11:34:01.508140 23105 catalog_manager.cc:5495] T 55d2acb48bd04b95913aa61fbc158fca P b53ea98f7b4b4a828f90fc4578efe84a reported cstate change: term changed from 0 to 1, leader changed from <none> to b53ea98f7b4b4a828f90fc4578efe84a (127.8.18.193). New cstate: current_term: 1 leader_uuid: "b53ea98f7b4b4a828f90fc4578efe84a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b53ea98f7b4b4a828f90fc4578efe84a" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 41937 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "abbaeefbabb941d8ae31fe6e637fdbc0" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 44985 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52042505dd7f42e990ee5684ada73014" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 43839 } health_report { overall_health: UNKNOWN } } }
W20260810 11:34:01.517056 23295 tablet.cc:2369] T 55d2acb48bd04b95913aa61fbc158fca P b53ea98f7b4b4a828f90fc4578efe84a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 11:34:01.531908 23578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:34:01.525943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45658 (local address 127.8.18.193:41937)
0810 11:34:01.526196 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0810 11:34:01.526201 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:34:01.526222 (+ 21us) server_negotiation.cc:407] Connection header received
0810 11:34:01.526321 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:34:01.526325 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:34:01.526408 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:34:01.526549 (+ 141us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 11:34:01.526970 (+ 421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:34:01.527784 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:34:01.529253 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:34:01.529842 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:34:01.529955 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:34:01.530553 (+ 598us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 11:34:01.531178 (+ 625us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 11:34:01.531311 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0810 11:34:01.531472 (+ 161us) server_negotiation.cc:299] Negotiation successful
0810 11:34:01.531635 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260810 11:34:01.541951 23383 raft_consensus.cc:1261] T 55d2acb48bd04b95913aa61fbc158fca P abbaeefbabb941d8ae31fe6e637fdbc0 [term 1 FOLLOWER]: Refusing update from remote peer b53ea98f7b4b4a828f90fc4578efe84a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 11:34:01.541953 23517 raft_consensus.cc:1261] T 55d2acb48bd04b95913aa61fbc158fca P 52042505dd7f42e990ee5684ada73014 [term 1 FOLLOWER]: Refusing update from remote peer b53ea98f7b4b4a828f90fc4578efe84a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 11:34:01.543211 23593 consensus_queue.cc:1035] T 55d2acb48bd04b95913aa61fbc158fca P b53ea98f7b4b4a828f90fc4578efe84a [LEADER]: Connected to new peer: Peer: permanent_uuid: "abbaeefbabb941d8ae31fe6e637fdbc0" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 44985 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 11:34:01.543672 23588 consensus_queue.cc:1035] T 55d2acb48bd04b95913aa61fbc158fca P b53ea98f7b4b4a828f90fc4578efe84a [LEADER]: Connected to new peer: Peer: permanent_uuid: "52042505dd7f42e990ee5684ada73014" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 43839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260810 11:34:01.553226 23430 tablet.cc:2369] T 55d2acb48bd04b95913aa61fbc158fca P abbaeefbabb941d8ae31fe6e637fdbc0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 11:34:01.557271 23595 mvcc.cc:205] Tried to move back new op lower bound from 7316937283743956992 to 7316937283601457152. Current Snapshot: MvccSnapshot[applied={T|T < 7316937283743956992}]
I20260810 11:34:01.575409 23104 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:48906:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 11:34:01.575788 23104 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:48906:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 11:34:01.580339 23104 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 99954a220cf4425b894a741dd117548a: Sending DeleteTablet for 3 replicas of tablet 55d2acb48bd04b95913aa61fbc158fca
I20260810 11:34:01.582000 23363 tablet_service.cc:1514] Processing DeleteTablet for tablet 55d2acb48bd04b95913aa61fbc158fca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:34:01 UTC) from {username='kudu', principal='kudu/127.8.18.254@KRBTEST.COM'} at 127.0.0.1:43470
I20260810 11:34:01.582005 23227 tablet_service.cc:1514] Processing DeleteTablet for tablet 55d2acb48bd04b95913aa61fbc158fca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:34:01 UTC) from {username='kudu', principal='kudu/127.8.18.254@KRBTEST.COM'} at 127.0.0.1:45656
I20260810 11:34:01.583120 23497 tablet_service.cc:1514] Processing DeleteTablet for tablet 55d2acb48bd04b95913aa61fbc158fca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:34:01 UTC) from {username='kudu', principal='kudu/127.8.18.254@KRBTEST.COM'} at 127.0.0.1:34824
I20260810 11:34:01.583386 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 23143
I20260810 11:34:01.584489 23607 tablet_replica.cc:330] T 55d2acb48bd04b95913aa61fbc158fca P 52042505dd7f42e990ee5684ada73014: stopping tablet replica
I20260810 11:34:01.585016 23607 raft_consensus.cc:2229] T 55d2acb48bd04b95913aa61fbc158fca P 52042505dd7f42e990ee5684ada73014 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 11:34:01.585639 23607 raft_consensus.cc:2258] T 55d2acb48bd04b95913aa61fbc158fca P 52042505dd7f42e990ee5684ada73014 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:34:01.586526 23605 tablet_replica.cc:330] T 55d2acb48bd04b95913aa61fbc158fca P abbaeefbabb941d8ae31fe6e637fdbc0: stopping tablet replica
I20260810 11:34:01.587054 23605 raft_consensus.cc:2229] T 55d2acb48bd04b95913aa61fbc158fca P abbaeefbabb941d8ae31fe6e637fdbc0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 11:34:01.587666 23605 raft_consensus.cc:2258] T 55d2acb48bd04b95913aa61fbc158fca P abbaeefbabb941d8ae31fe6e637fdbc0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:34:01.589689 23606 tablet_replica.cc:330] T 55d2acb48bd04b95913aa61fbc158fca P b53ea98f7b4b4a828f90fc4578efe84a: stopping tablet replica
I20260810 11:34:01.590199 23606 raft_consensus.cc:2229] T 55d2acb48bd04b95913aa61fbc158fca P b53ea98f7b4b4a828f90fc4578efe84a [term 1 LEADER]: Raft consensus shutting down.
I20260810 11:34:01.590327 23605 ts_tablet_manager.cc:1905] T 55d2acb48bd04b95913aa61fbc158fca P abbaeefbabb941d8ae31fe6e637fdbc0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:34:01.591003 23606 raft_consensus.cc:2258] T 55d2acb48bd04b95913aa61fbc158fca P b53ea98f7b4b4a828f90fc4578efe84a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:34:01.592651 23607 ts_tablet_manager.cc:1905] T 55d2acb48bd04b95913aa61fbc158fca P 52042505dd7f42e990ee5684ada73014: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:34:01.597359 23605 ts_tablet_manager.cc:1918] T 55d2acb48bd04b95913aa61fbc158fca P abbaeefbabb941d8ae31fe6e637fdbc0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 11:34:01.597594 23605 log.cc:1196] T 55d2acb48bd04b95913aa61fbc158fca P abbaeefbabb941d8ae31fe6e637fdbc0: Deleting WAL directory at /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/ts-1/wal/wals/55d2acb48bd04b95913aa61fbc158fca
I20260810 11:34:01.598167 23605 ts_tablet_manager.cc:1939] T 55d2acb48bd04b95913aa61fbc158fca P abbaeefbabb941d8ae31fe6e637fdbc0: Deleting consensus metadata
I20260810 11:34:01.600322 23093 catalog_manager.cc:4841] TS abbaeefbabb941d8ae31fe6e637fdbc0 (127.8.18.194:44985): tablet 55d2acb48bd04b95913aa61fbc158fca (table test-table [id=8016185a0d6d47f19894516eb0977b9c]) successfully deleted
I20260810 11:34:01.600745 23606 ts_tablet_manager.cc:1905] T 55d2acb48bd04b95913aa61fbc158fca P b53ea98f7b4b4a828f90fc4578efe84a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:34:01.605151 23578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:34:01.584469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45666 (local address 127.8.18.193:41937)
0810 11:34:01.585297 (+ 828us) server_negotiation.cc:206] Beginning negotiation
0810 11:34:01.585308 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 11:34:01.585333 (+ 25us) server_negotiation.cc:407] Connection header received
0810 11:34:01.585438 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:34:01.585447 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:34:01.585552 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:34:01.585691 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:34:01.588504 (+ 2813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:34:01.589625 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:34:01.591623 (+ 1998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:34:01.592290 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:34:01.592434 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:34:01.596364 (+ 3930us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:34:01.596383 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:34:01.596403 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:34:01.596461 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:34:01.600791 (+ 4330us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:34:01.602470 (+ 1679us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:34:01.602487 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:34:01.602492 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:34:01.602715 (+ 223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:34:01.603218 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:34:01.603225 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:34:01.603228 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:34:01.603749 (+ 521us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:34:01.604120 (+ 371us) server_negotiation.cc:1035] Waiting for connection context
0810 11:34:01.604486 (+ 366us) server_negotiation.cc:299] Negotiation successful
0810 11:34:01.604769 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":646}
I20260810 11:34:01.606190 23289 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:34:01.606891 23607 ts_tablet_manager.cc:1918] T 55d2acb48bd04b95913aa61fbc158fca P 52042505dd7f42e990ee5684ada73014: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 11:34:01.607096 23607 log.cc:1196] T 55d2acb48bd04b95913aa61fbc158fca P 52042505dd7f42e990ee5684ada73014: Deleting WAL directory at /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/ts-2/wal/wals/55d2acb48bd04b95913aa61fbc158fca
I20260810 11:34:01.607779 23607 ts_tablet_manager.cc:1939] T 55d2acb48bd04b95913aa61fbc158fca P 52042505dd7f42e990ee5684ada73014: Deleting consensus metadata
I20260810 11:34:01.610051 23092 catalog_manager.cc:4841] TS 52042505dd7f42e990ee5684ada73014 (127.8.18.195:43839): tablet 55d2acb48bd04b95913aa61fbc158fca (table test-table [id=8016185a0d6d47f19894516eb0977b9c]) successfully deleted
I20260810 11:34:01.716751 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 23143
W20260810 11:34:01.740013 23092 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.18.193:41937 (error 108)
I20260810 11:34:01.740401 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 23298
I20260810 11:34:01.742420 23581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:34:01.741852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.193:41937 (local address 127.0.0.1:45678)
0810 11:34:01.742083 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0810 11:34:01.742215 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.193:41937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
W20260810 11:34:01.742904 23092 catalog_manager.cc:4584] TS b53ea98f7b4b4a828f90fc4578efe84a (127.8.18.193:41937): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 55d2acb48bd04b95913aa61fbc158fca: Network error: Client connection negotiation failed: client connection to 127.8.18.193:41937: connect: Connection refused (error 111)
I20260810 11:34:01.758378 23579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:34:01.741507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43482 (local address 127.8.18.194:44985)
0810 11:34:01.741724 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0810 11:34:01.741732 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:34:01.741772 (+ 40us) server_negotiation.cc:407] Connection header received
0810 11:34:01.741841 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:34:01.741846 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:34:01.741930 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:34:01.742058 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:34:01.744069 (+ 2011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:34:01.745009 (+ 940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:34:01.746663 (+ 1654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:34:01.747589 (+ 926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:34:01.747743 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:34:01.750965 (+ 3222us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:34:01.750991 (+ 26us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:34:01.751010 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:34:01.751058 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:34:01.754529 (+ 3471us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:34:01.755520 (+ 991us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:34:01.755528 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:34:01.755530 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:34:01.755713 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:34:01.756287 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:34:01.756291 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:34:01.756293 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:34:01.756771 (+ 478us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:34:01.757043 (+ 272us) server_negotiation.cc:1035] Waiting for connection context
0810 11:34:01.757639 (+ 596us) server_negotiation.cc:299] Negotiation successful
0810 11:34:01.757969 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260810 11:34:01.759277 23424 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:34:01.796738 23581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:34:01.796111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.193:41937 (local address 127.0.0.1:45692)
0810 11:34:01.796386 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0810 11:34:01.796523 (+ 137us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.193:41937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105}
I20260810 11:34:01.798111 23581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:34:01.797639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.193:41937 (local address 127.0.0.1:45704)
0810 11:34:01.797806 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0810 11:34:01.797901 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.193:41937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260810 11:34:01.855785 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 23298
I20260810 11:34:01.858863 23581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:34:01.858346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.193:41937 (local address 127.0.0.1:45712)
0810 11:34:01.858578 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0810 11:34:01.858693 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.193:41937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260810 11:34:01.859903 23581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:34:01.859555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.193:41937 (local address 127.0.0.1:45718)
0810 11:34:01.859725 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0810 11:34:01.859796 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.193:41937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260810 11:34:01.876673 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 23433
I20260810 11:34:01.893595 23580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:34:01.877774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34836 (local address 127.8.18.195:43839)
0810 11:34:01.877995 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0810 11:34:01.878003 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:34:01.878096 (+ 93us) server_negotiation.cc:407] Connection header received
0810 11:34:01.878370 (+ 274us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:34:01.878381 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:34:01.878460 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:34:01.878579 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:34:01.879989 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:34:01.880786 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:34:01.882627 (+ 1841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:34:01.883242 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:34:01.883435 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:34:01.886846 (+ 3411us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:34:01.886868 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:34:01.886885 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:34:01.886932 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:34:01.890138 (+ 3206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:34:01.891070 (+ 932us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:34:01.891079 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:34:01.891082 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:34:01.891249 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:34:01.891738 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:34:01.891742 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:34:01.891744 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:34:01.892158 (+ 414us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:34:01.892489 (+ 331us) server_negotiation.cc:1035] Waiting for connection context
0810 11:34:01.892874 (+ 385us) server_negotiation.cc:299] Negotiation successful
0810 11:34:01.893143 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260810 11:34:01.894371 23558 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:34:01.940805 23581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:34:01.940229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.193:41937 (local address 127.0.0.1:45730)
0810 11:34:01.940503 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0810 11:34:01.940610 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.193:41937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120}
I20260810 11:34:01.942260 23581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:34:01.941812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.193:41937 (local address 127.0.0.1:45742)
0810 11:34:01.942040 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0810 11:34:01.942126 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.193:41937: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127}
I20260810 11:34:01.994942 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 23433
I20260810 11:34:02.016037 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 23072
I20260810 11:34:02.017463 23136 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:34:02.140360 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 23072
I20260810 11:34:02.163056 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.18.254:43799
--webserver_interface=127.8.18.254
--webserver_port=40949
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.254.keytab
--principal=kudu/127.8.18.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.18.212:35253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.18.254:43799
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:34:02.531649 23613 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:34:02.531982 23613 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:34:02.532153 23613 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:34:02.539932 23613 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 11:34:02.540079 23613 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:34:02.540186 23613 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:34:02.540284 23613 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 11:34:02.540372 23613 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 11:34:02.551023 23613 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.18.212:35253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.18.254:43799
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.254.keytab
--principal=kudu/127.8.18.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.18.254:43799
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.18.254
--webserver_port=40949
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:34:02.552573 23613 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:34:02.554039 23613 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:34:02.567158 23619 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:34:02.567138 23621 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:34:02.567303 23618 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:34:02.568115 23613 server_base.cc:1034] running on GCE node
I20260810 11:34:02.569052 23613 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:34:02.570744 23613 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:34:02.571985 23613 hybrid_clock.cc:648] HybridClock initialized: now 1786361642571908 us; error 63 us; skew 500 ppm
I20260810 11:34:02.572403 23613 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:34:02 dist-test-slave-srgv krb5kdc[22509](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361642, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:34:02.576164 23613 init.cc:377] Logged in from keytab as kudu/127.8.18.254@KRBTEST.COM (short username kudu)
I20260810 11:34:02.577996 23613 webserver.cc:460] Webserver started at http://127.8.18.254:40949/ using document root <none> and password file <none>
I20260810 11:34:02.578827 23613 fs_manager.cc:359] Metadata directory not provided
I20260810 11:34:02.579005 23613 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:34:02.585649 23613 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.007s sys 0.000s
I20260810 11:34:02.589020 23628 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:34:02.590612 23613 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 11:34:02.590857 23613 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/master-0/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/master-0/wal
uuid: "99954a220cf4425b894a741dd117548a"
format_stamp: "Formatted at 2026-08-10 11:33:56 on dist-test-slave-srgv"
server_key: "dd0a6aa35d36a6fdd867b117088c8f9a"
server_key_iv: "6239f1ab4e6711ccf45bd557db90dd00"
server_key_version: "encryptionkey@0"
I20260810 11:34:02.592006 23613 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:34:02.617189 23613 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:34:02.622368 23613 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:34:02.622711 23613 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:34:02.644079 23613 rpc_server.cc:307] RPC server started. Bound to: 127.8.18.254:43799
I20260810 11:34:02.644102 23679 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.18.254:43799 every 8 connection(s)
I20260810 11:34:02.645900 23613 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
I20260810 11:34:02.653498 23680 sys_catalog.cc:263] Verifying existing consensus state
I20260810 11:34:02.655344 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 23613
I20260810 11:34:02.656235 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.18.193:41937
--local_ip_for_outbound_sockets=127.8.18.193
--tserver_master_addrs=127.8.18.254:43799
--webserver_port=43503
--webserver_interface=127.8.18.193
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.193.keytab
--principal=kudu/127.8.18.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.18.212:35253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 11:34:02.657859 23680 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 99954a220cf4425b894a741dd117548a: Bootstrap starting.
I20260810 11:34:02.685736 23680 log.cc:824] T 00000000000000000000000000000000 P 99954a220cf4425b894a741dd117548a: Log is configured to *not* fsync() on all Append() calls
I20260810 11:34:02.709645 23680 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 99954a220cf4425b894a741dd117548a: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 11:34:02.710409 23680 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 99954a220cf4425b894a741dd117548a: Bootstrap complete.
I20260810 11:34:02.725174 23680 raft_consensus.cc:348] T 00000000000000000000000000000000 P 99954a220cf4425b894a741dd117548a [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99954a220cf4425b894a741dd117548a" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 43799 } }
I20260810 11:34:02.726812 23680 raft_consensus.cc:729] T 00000000000000000000000000000000 P 99954a220cf4425b894a741dd117548a [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99954a220cf4425b894a741dd117548a, State: Initialized, Role: FOLLOWER
I20260810 11:34:02.727682 23680 consensus_queue.cc:260] T 00000000000000000000000000000000 P 99954a220cf4425b894a741dd117548a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99954a220cf4425b894a741dd117548a" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 43799 } }
I20260810 11:34:02.728006 23680 raft_consensus.cc:388] T 00000000000000000000000000000000 P 99954a220cf4425b894a741dd117548a [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 11:34:02.728170 23680 raft_consensus.cc:482] T 00000000000000000000000000000000 P 99954a220cf4425b894a741dd117548a [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 11:34:02.728380 23680 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 99954a220cf4425b894a741dd117548a [term 2 FOLLOWER]: Advancing to term 3
I20260810 11:34:02.735347 23680 raft_consensus.cc:504] T 00000000000000000000000000000000 P 99954a220cf4425b894a741dd117548a [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99954a220cf4425b894a741dd117548a" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 43799 } }
I20260810 11:34:02.736303 23680 leader_election.cc:302] T 00000000000000000000000000000000 P 99954a220cf4425b894a741dd117548a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 99954a220cf4425b894a741dd117548a; no voters:
I20260810 11:34:02.737197 23680 leader_election.cc:288] T 00000000000000000000000000000000 P 99954a220cf4425b894a741dd117548a [CANDIDATE]: Term 3 election: Requested vote from peers
I20260810 11:34:02.737455 23684 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 99954a220cf4425b894a741dd117548a [term 3 FOLLOWER]: Leader election won for term 3
I20260810 11:34:02.738982 23684 raft_consensus.cc:686] T 00000000000000000000000000000000 P 99954a220cf4425b894a741dd117548a [term 3 LEADER]: Becoming Leader. State: Replica: 99954a220cf4425b894a741dd117548a, State: Running, Role: LEADER
I20260810 11:34:02.739607 23684 consensus_queue.cc:237] T 00000000000000000000000000000000 P 99954a220cf4425b894a741dd117548a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99954a220cf4425b894a741dd117548a" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 43799 } }
I20260810 11:34:02.739811 23680 sys_catalog.cc:564] T 00000000000000000000000000000000 P 99954a220cf4425b894a741dd117548a [sys.catalog]: configured and running, proceeding with master startup.
I20260810 11:34:02.742455 23686 sys_catalog.cc:455] T 00000000000000000000000000000000 P 99954a220cf4425b894a741dd117548a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 99954a220cf4425b894a741dd117548a. Latest consensus state: current_term: 3 leader_uuid: "99954a220cf4425b894a741dd117548a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99954a220cf4425b894a741dd117548a" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 43799 } } }
I20260810 11:34:02.742947 23686 sys_catalog.cc:458] T 00000000000000000000000000000000 P 99954a220cf4425b894a741dd117548a [sys.catalog]: This master's current role is: LEADER
I20260810 11:34:02.742455 23685 sys_catalog.cc:455] T 00000000000000000000000000000000 P 99954a220cf4425b894a741dd117548a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "99954a220cf4425b894a741dd117548a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99954a220cf4425b894a741dd117548a" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 43799 } } }
I20260810 11:34:02.743629 23685 sys_catalog.cc:458] T 00000000000000000000000000000000 P 99954a220cf4425b894a741dd117548a [sys.catalog]: This master's current role is: LEADER
I20260810 11:34:02.745388 23690 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 11:34:02.757860 23690 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 11:34:02.761538 23690 catalog_manager.cc:1259] Loaded cluster ID: b7dd071c324847619421b3caaec5f6e3
I20260810 11:34:02.761826 23690 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260810 11:34:02.765293 23690 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f781dd1e9ad google::LogMessage::Fail() at ??:0
@ 0x7f781dd2188a google::LogMessage::SendToLog() at ??:0
@ 0x7f781dd1e4de google::LogMessage::Flush() at ??:0
@ 0x7f781dd1fa19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f7833751512 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7f783374e976 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f78337d6610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7f78337d6439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f782096a345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f782096d91c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f782096d799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f7820948eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f7823c226db start_thread at ??:0
@ 0x7f781cbb861f clone at ??:0
W20260810 11:34:03.104444 23682 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:34:03.104838 23682 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:34:03.105026 23682 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:34:03.113543 23682 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:34:03.113711 23682 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:34:03.113914 23682 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.18.193
I20260810 11:34:03.124951 23682 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.18.212:35253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.193.keytab
--principal=kudu/127.8.18.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.18.193:41937
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.18.193
--webserver_port=43503
--tserver_master_addrs=127.8.18.254:43799
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.18.193
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:34:03.126554 23682 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:34:03.128062 23682 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:34:03.143321 23726 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:34:03.143517 23723 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:34:03.143364 23724 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:34:03.144415 23682 server_base.cc:1034] running on GCE node
I20260810 11:34:03.145154 23682 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:34:03.146435 23682 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:34:03.147750 23682 hybrid_clock.cc:648] HybridClock initialized: now 1786361643147669 us; error 65 us; skew 500 ppm
I20260810 11:34:03.148159 23682 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:34:03 dist-test-slave-srgv krb5kdc[22509](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361643, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:34:03.152150 23682 init.cc:377] Logged in from keytab as kudu/127.8.18.193@KRBTEST.COM (short username kudu)
I20260810 11:34:03.153976 23682 webserver.cc:460] Webserver started at http://127.8.18.193:43503/ using document root <none> and password file <none>
I20260810 11:34:03.154901 23682 fs_manager.cc:359] Metadata directory not provided
I20260810 11:34:03.155081 23682 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:34:03.161829 23682 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.007s sys 0.000s
I20260810 11:34:03.165400 23733 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:34:03.167052 23682 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 11:34:03.167273 23682 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/ts-0/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/ts-0/wal
uuid: "b53ea98f7b4b4a828f90fc4578efe84a"
format_stamp: "Formatted at 2026-08-10 11:33:56 on dist-test-slave-srgv"
server_key: "432487eb597c21bb93993d7fff75c65d"
server_key_iv: "fd74c204c2a17562810d2d473e2f02f4"
server_key_version: "encryptionkey@0"
I20260810 11:34:03.168444 23682 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:34:03.200989 23682 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:34:03.206560 23682 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:34:03.206931 23682 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:34:03.208209 23682 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:34:03.210520 23740 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
W20260810 11:34:03.216876 23740 ts_tablet_manager.cc:1852] T 55d2acb48bd04b95913aa61fbc158fca P b53ea98f7b4b4a828f90fc4578efe84a: Tablet Manager startup: Rolling forward tablet deletion of type TABLET_DATA_DELETED
I20260810 11:34:03.218080 23740 ts_tablet_manager.cc:1905] T 55d2acb48bd04b95913aa61fbc158fca P b53ea98f7b4b4a828f90fc4578efe84a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:34:03.226930 23740 ts_tablet_manager.cc:1918] T 55d2acb48bd04b95913aa61fbc158fca P b53ea98f7b4b4a828f90fc4578efe84a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 11:34:03.227222 23740 log.cc:1196] T 55d2acb48bd04b95913aa61fbc158fca P b53ea98f7b4b4a828f90fc4578efe84a: Deleting WAL directory at /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/ts-0/wal/wals/55d2acb48bd04b95913aa61fbc158fca
I20260810 11:34:03.227833 23740 ts_tablet_manager.cc:1939] T 55d2acb48bd04b95913aa61fbc158fca P b53ea98f7b4b4a828f90fc4578efe84a: Deleting consensus metadata
I20260810 11:34:03.228322 23682 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 0 live tablets)
I20260810 11:34:03.228494 23682 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.018s user 0.000s sys 0.001s
I20260810 11:34:03.228652 23682 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:34:03.228754 23682 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:34:03.267731 23682 rpc_server.cc:307] RPC server started. Bound to: 127.8.18.193:41937
I20260810 11:34:03.267784 23847 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.18.193:41937 every 8 connection(s)
I20260810 11:34:03.269381 23682 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/ts-0/data/info.pb
I20260810 11:34:03.275611 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 23682
I20260810 11:34:03.276207 23850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:34:03.273608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:43799 (local address 127.8.18.193:35821)
0810 11:34:03.274750 (+ 1142us) negotiation.cc:107] Waiting for socket to connect
0810 11:34:03.275026 (+ 276us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.254:43799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":852,"thread_start_us":583,"threads_started":1}
I20260810 11:34:03.276458 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.18.194:44985
--local_ip_for_outbound_sockets=127.8.18.194
--tserver_master_addrs=127.8.18.254:43799
--webserver_port=36647
--webserver_interface=127.8.18.194
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.194.keytab
--principal=kudu/127.8.18.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.18.212:35253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:34:03.277109 23848 heartbeater.cc:644] Failed to heartbeat to 127.8.18.254:43799 (0 consecutive failures): Network error: Failed to ping master at 127.8.18.254:43799: Client connection negotiation failed: client connection to 127.8.18.254:43799: connect: Connection refused (error 111)
I20260810 11:34:03.278323 23850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:34:03.277896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:43799 (local address 127.8.18.193:56327)
0810 11:34:03.278100 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0810 11:34:03.278185 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.254:43799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
I20260810 11:34:03.279539 23850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:34:03.279016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:43799 (local address 127.8.18.193:56349)
0810 11:34:03.279226 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0810 11:34:03.279343 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.254:43799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
W20260810 11:34:03.279848 23848 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260810 11:34:03.652197 23851 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:34:03.652536 23851 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:34:03.652724 23851 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:34:03.660831 23851 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:34:03.660979 23851 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:34:03.661183 23851 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.18.194
I20260810 11:34:03.671816 23851 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.18.212:35253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.194.keytab
--principal=kudu/127.8.18.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.18.194:44985
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.18.194
--webserver_port=36647
--tserver_master_addrs=127.8.18.254:43799
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.18.194
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:34:03.673506 23851 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:34:03.675113 23851 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:34:03.691112 23856 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:34:03.691035 23857 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:34:03.691073 23859 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:34:03.692130 23851 server_base.cc:1034] running on GCE node
I20260810 11:34:03.692948 23851 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:34:03.694257 23851 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:34:03.695521 23851 hybrid_clock.cc:648] HybridClock initialized: now 1786361643695430 us; error 86 us; skew 500 ppm
I20260810 11:34:03.695935 23851 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:34:03 dist-test-slave-srgv krb5kdc[22509](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361643, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:34:03.700101 23851 init.cc:377] Logged in from keytab as kudu/127.8.18.194@KRBTEST.COM (short username kudu)
I20260810 11:34:03.702020 23851 webserver.cc:460] Webserver started at http://127.8.18.194:36647/ using document root <none> and password file <none>
I20260810 11:34:03.702870 23851 fs_manager.cc:359] Metadata directory not provided
I20260810 11:34:03.703042 23851 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:34:03.709465 23851 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260810 11:34:03.713178 23866 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:34:03.715031 23851 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.003s
I20260810 11:34:03.715315 23851 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/ts-1/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/ts-1/wal
uuid: "abbaeefbabb941d8ae31fe6e637fdbc0"
format_stamp: "Formatted at 2026-08-10 11:33:57 on dist-test-slave-srgv"
server_key: "42d45e663bfadeb03076cdff4ecf7fef"
server_key_iv: "c89e0cbaa3e21eb63f117d46dd0343af"
server_key_version: "encryptionkey@0"
I20260810 11:34:03.716550 23851 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:34:03.752890 23851 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:34:03.758495 23851 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:34:03.758901 23851 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:34:03.760556 23851 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:34:03.762708 23851 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:34:03.762889 23851 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:34:03.763068 23851 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:34:03.763177 23851 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:34:03.805712 23851 rpc_server.cc:307] RPC server started. Bound to: 127.8.18.194:44985
I20260810 11:34:03.805817 23980 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.18.194:44985 every 8 connection(s)
I20260810 11:34:03.807596 23851 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/ts-1/data/info.pb
I20260810 11:34:03.810649 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 23851
I20260810 11:34:03.811326 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.18.195:43839
--local_ip_for_outbound_sockets=127.8.18.195
--tserver_master_addrs=127.8.18.254:43799
--webserver_port=35623
--webserver_interface=127.8.18.195
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.195.keytab
--principal=kudu/127.8.18.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.18.212:35253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 11:34:03.814637 23983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:34:03.811954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:43799 (local address 127.8.18.194:51413)
0810 11:34:03.813057 (+ 1103us) negotiation.cc:107] Waiting for socket to connect
0810 11:34:03.813438 (+ 381us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.254:43799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":737,"thread_start_us":476,"threads_started":1}
W20260810 11:34:03.815729 23981 heartbeater.cc:644] Failed to heartbeat to 127.8.18.254:43799 (0 consecutive failures): Network error: Failed to ping master at 127.8.18.254:43799: Client connection negotiation failed: client connection to 127.8.18.254:43799: connect: Connection refused (error 111)
I20260810 11:34:03.817025 23983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:34:03.816567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:43799 (local address 127.8.18.194:34193)
0810 11:34:03.816786 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0810 11:34:03.816882 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.254:43799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260810 11:34:03.818348 23983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:34:03.817944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:43799 (local address 127.8.18.194:57327)
0810 11:34:03.818109 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0810 11:34:03.818200 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.254:43799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260810 11:34:03.818727 23981 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260810 11:34:04.188970 23984 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:34:04.189303 23984 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:34:04.189467 23984 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:34:04.197551 23984 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:34:04.197691 23984 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:34:04.197817 23984 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.18.195
I20260810 11:34:04.208920 23984 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.18.212:35253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.195.keytab
--principal=kudu/127.8.18.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.18.195:43839
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.18.195
--webserver_port=35623
--tserver_master_addrs=127.8.18.254:43799
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.18.195
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:34:04.210402 23984 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:34:04.211937 23984 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:34:04.227762 23992 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:34:04.227797 23990 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:34:04.227833 23989 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:34:04.228607 23984 server_base.cc:1034] running on GCE node
I20260810 11:34:04.229342 23984 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:34:04.230536 23984 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:34:04.231817 23984 hybrid_clock.cc:648] HybridClock initialized: now 1786361644231742 us; error 68 us; skew 500 ppm
I20260810 11:34:04.232190 23984 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:34:04 dist-test-slave-srgv krb5kdc[22509](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361644, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:34:04.239859 23984 init.cc:377] Logged in from keytab as kudu/127.8.18.195@KRBTEST.COM (short username kudu)
I20260810 11:34:04.241865 23984 webserver.cc:460] Webserver started at http://127.8.18.195:35623/ using document root <none> and password file <none>
I20260810 11:34:04.242689 23984 fs_manager.cc:359] Metadata directory not provided
I20260810 11:34:04.242856 23984 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:34:04.249308 23984 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260810 11:34:04.253010 23999 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:34:04.254837 23984 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260810 11:34:04.255102 23984 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/ts-2/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/ts-2/wal
uuid: "52042505dd7f42e990ee5684ada73014"
format_stamp: "Formatted at 2026-08-10 11:33:58 on dist-test-slave-srgv"
server_key: "5de592d87e7c1bb273f31396872e2cd2"
server_key_iv: "50211b8d617be8277b70ee6472c7f469"
server_key_version: "encryptionkey@0"
I20260810 11:34:04.256340 23984 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:34:04.281948 24004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:34:04.280683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:43799 (local address 127.8.18.193:44005)
0810 11:34:04.281300 (+ 617us) negotiation.cc:107] Waiting for socket to connect
0810 11:34:04.281599 (+ 299us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.254:43799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":477,"thread_start_us":321,"threads_started":1}
I20260810 11:34:04.282660 23984 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:34:04.288110 23984 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:34:04.288479 23984 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:34:04.289812 23984 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:34:04.291659 23984 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:34:04.291849 23984 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:34:04.292023 23984 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:34:04.292130 23984 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:34:04.335747 23984 rpc_server.cc:307] RPC server started. Bound to: 127.8.18.195:43839
I20260810 11:34:04.335865 24117 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.18.195:43839 every 8 connection(s)
I20260810 11:34:04.337652 23984 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786361401922362-8267-0/minicluster-data/ts-2/data/info.pb
I20260810 11:34:04.343470 24120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:34:04.341095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:43799 (local address 127.8.18.195:53567)
0810 11:34:04.342033 (+ 938us) negotiation.cc:107] Waiting for socket to connect
0810 11:34:04.342342 (+ 309us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.254:43799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":641,"thread_start_us":415,"threads_started":1}
W20260810 11:34:04.344462 24118 heartbeater.cc:644] Failed to heartbeat to 127.8.18.254:43799 (0 consecutive failures): Network error: Failed to ping master at 127.8.18.254:43799: Client connection negotiation failed: client connection to 127.8.18.254:43799: connect: Connection refused (error 111)
I20260810 11:34:04.344874 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 23984
I20260810 11:34:04.345773 24120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:34:04.345164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:43799 (local address 127.8.18.195:47259)
0810 11:34:04.345398 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0810 11:34:04.345572 (+ 174us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.254:43799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260810 11:34:04.346930 24120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:34:04.346493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:43799 (local address 127.8.18.195:52301)
0810 11:34:04.346674 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0810 11:34:04.346781 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.254:43799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260810 11:34:04.347314 24118 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260810 11:34:04.820536 24122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:34:04.819502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:43799 (local address 127.8.18.194:40149)
0810 11:34:04.820148 (+ 646us) negotiation.cc:107] Waiting for socket to connect
0810 11:34:04.820260 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.254:43799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":303,"threads_started":1}
I20260810 11:34:05.284243 24123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:34:05.283175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:43799 (local address 127.8.18.193:34935)
0810 11:34:05.283822 (+ 647us) negotiation.cc:107] Waiting for socket to connect
0810 11:34:05.283960 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.254:43799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":499,"thread_start_us":287,"threads_started":1}
I20260810 11:34:05.348928 24124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:34:05.348011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:43799 (local address 127.8.18.195:37467)
0810 11:34:05.348545 (+ 534us) negotiation.cc:107] Waiting for socket to connect
0810 11:34:05.348667 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.254:43799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":414,"thread_start_us":265,"threads_started":1}
I20260810 11:34:05.822790 24125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:34:05.821839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:43799 (local address 127.8.18.194:57209)
0810 11:34:05.822480 (+ 641us) negotiation.cc:107] Waiting for socket to connect
0810 11:34:05.822596 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.254:43799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":511,"thread_start_us":287,"threads_started":1}
I20260810 11:34:06.286516 24126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:34:06.285656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:43799 (local address 127.8.18.193:57161)
0810 11:34:06.286202 (+ 546us) negotiation.cc:107] Waiting for socket to connect
0810 11:34:06.286309 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.254:43799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":432,"thread_start_us":284,"threads_started":1}
I20260810 11:34:06.351084 24127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:34:06.350195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:43799 (local address 127.8.18.195:37365)
0810 11:34:06.350785 (+ 590us) negotiation.cc:107] Waiting for socket to connect
0810 11:34:06.350886 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.254:43799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":471,"thread_start_us":300,"threads_started":1}
I20260810 11:34:06.824697 24128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:34:06.823784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:43799 (local address 127.8.18.194:44069)
0810 11:34:06.824379 (+ 595us) negotiation.cc:107] Waiting for socket to connect
0810 11:34:06.824487 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.254:43799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":344,"threads_started":1}
I20260810 11:34:07.288722 24129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:34:07.287849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:43799 (local address 127.8.18.193:49337)
0810 11:34:07.288429 (+ 580us) negotiation.cc:107] Waiting for socket to connect
0810 11:34:07.288533 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.254:43799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":300,"threads_started":1}
I20260810 11:34:07.353047 24130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:34:07.352224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:43799 (local address 127.8.18.195:41603)
0810 11:34:07.352757 (+ 533us) negotiation.cc:107] Waiting for socket to connect
0810 11:34:07.352850 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.254:43799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":424,"thread_start_us":293,"threads_started":1}
I20260810 11:34:07.826704 24131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:34:07.825872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:43799 (local address 127.8.18.194:44945)
0810 11:34:07.826402 (+ 530us) negotiation.cc:107] Waiting for socket to connect
0810 11:34:07.826491 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.254:43799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":421,"thread_start_us":262,"threads_started":1}
I20260810 11:34:08.290863 24132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:34:08.289998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:43799 (local address 127.8.18.193:45157)
0810 11:34:08.290576 (+ 578us) negotiation.cc:107] Waiting for socket to connect
0810 11:34:08.290673 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.254:43799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":434,"thread_start_us":292,"threads_started":1}
I20260810 11:34:08.355069 24133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:34:08.354247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:43799 (local address 127.8.18.195:43417)
0810 11:34:08.354784 (+ 537us) negotiation.cc:107] Waiting for socket to connect
0810 11:34:08.354879 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.254:43799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":418,"thread_start_us":269,"threads_started":1}
I20260810 11:34:08.828887 24134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:34:08.828009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:43799 (local address 127.8.18.194:46175)
0810 11:34:08.828592 (+ 583us) negotiation.cc:107] Waiting for socket to connect
0810 11:34:08.828689 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.254:43799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":473,"thread_start_us":277,"threads_started":1}
I20260810 11:34:09.293670 24135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:34:09.292230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:43799 (local address 127.8.18.193:48923)
0810 11:34:09.293229 (+ 999us) negotiation.cc:107] Waiting for socket to connect
0810 11:34:09.293359 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.254:43799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":806,"thread_start_us":425,"threads_started":1}
I20260810 11:34:09.357165 24136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:34:09.356334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:43799 (local address 127.8.18.195:41165)
0810 11:34:09.356888 (+ 554us) negotiation.cc:107] Waiting for socket to connect
0810 11:34:09.356983 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.254:43799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":445,"thread_start_us":273,"threads_started":1}
I20260810 11:34:09.830940 24137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:34:09.830083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:43799 (local address 127.8.18.194:35237)
0810 11:34:09.830613 (+ 530us) negotiation.cc:107] Waiting for socket to connect
0810 11:34:09.830752 (+ 139us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.254:43799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":416,"thread_start_us":278,"threads_started":1}
I20260810 11:34:10.296064 24138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:34:10.295029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:43799 (local address 127.8.18.193:46871)
0810 11:34:10.295746 (+ 717us) negotiation.cc:107] Waiting for socket to connect
0810 11:34:10.295839 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.254:43799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":574,"thread_start_us":353,"threads_started":1}
I20260810 11:34:10.359157 24139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:34:10.358234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:43799 (local address 127.8.18.195:56145)
0810 11:34:10.358870 (+ 636us) negotiation.cc:107] Waiting for socket to connect
0810 11:34:10.358975 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.254:43799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":525,"thread_start_us":394,"threads_started":1}
I20260810 11:34:10.833479 24140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:34:10.832296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:43799 (local address 127.8.18.194:35265)
0810 11:34:10.832965 (+ 669us) negotiation.cc:107] Waiting for socket to connect
0810 11:34:10.833151 (+ 186us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.254:43799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":539,"thread_start_us":324,"threads_started":1}
I20260810 11:34:11.298238 24141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:34:11.297311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:43799 (local address 127.8.18.193:59043)
0810 11:34:11.297901 (+ 590us) negotiation.cc:107] Waiting for socket to connect
0810 11:34:11.297997 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.254:43799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":473,"thread_start_us":338,"threads_started":1}
I20260810 11:34:11.361351 24142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:34:11.360290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:43799 (local address 127.8.18.195:39939)
0810 11:34:11.360937 (+ 647us) negotiation.cc:107] Waiting for socket to connect
0810 11:34:11.361049 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.254:43799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":502,"thread_start_us":267,"threads_started":1}
I20260810 11:34:11.835458 24143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:34:11.834575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:43799 (local address 127.8.18.194:46027)
0810 11:34:11.835168 (+ 593us) negotiation.cc:107] Waiting for socket to connect
0810 11:34:11.835262 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.254:43799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":432,"thread_start_us":285,"threads_started":1}
I20260810 11:34:12.300523 24144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:34:12.299465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:43799 (local address 127.8.18.193:33955)
0810 11:34:12.300159 (+ 694us) negotiation.cc:107] Waiting for socket to connect
0810 11:34:12.300309 (+ 150us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.254:43799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":488,"thread_start_us":330,"threads_started":1}
I20260810 11:34:12.363217 24145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:34:12.362415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:43799 (local address 127.8.18.195:44499)
0810 11:34:12.362909 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0810 11:34:12.362994 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.254:43799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":388,"thread_start_us":271,"threads_started":1}
I20260810 11:34:12.837591 24147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:34:12.836551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:43799 (local address 127.8.18.194:39189)
0810 11:34:12.837212 (+ 661us) negotiation.cc:107] Waiting for socket to connect
0810 11:34:12.837359 (+ 147us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.254:43799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":539,"thread_start_us":364,"threads_started":1}
I20260810 11:34:13.302771 24148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:34:13.301933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:43799 (local address 127.8.18.193:39351)
0810 11:34:13.302484 (+ 551us) negotiation.cc:107] Waiting for socket to connect
0810 11:34:13.302585 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.254:43799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":429,"thread_start_us":300,"threads_started":1}
I20260810 11:34:13.365259 24149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:34:13.364315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:43799 (local address 127.8.18.195:43427)
0810 11:34:13.364990 (+ 675us) negotiation.cc:107] Waiting for socket to connect
0810 11:34:13.365075 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.254:43799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":549,"thread_start_us":404,"threads_started":1}
I20260810 11:34:13.839697 24150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:34:13.838742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:43799 (local address 127.8.18.194:56981)
0810 11:34:13.839402 (+ 660us) negotiation.cc:107] Waiting for socket to connect
0810 11:34:13.839501 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.254:43799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":537,"thread_start_us":387,"threads_started":1}
I20260810 11:34:14.304946 24151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:34:14.304043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:43799 (local address 127.8.18.193:37397)
0810 11:34:14.304641 (+ 598us) negotiation.cc:107] Waiting for socket to connect
0810 11:34:14.304745 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.254:43799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":479,"thread_start_us":347,"threads_started":1}
I20260810 11:34:14.367065 24152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:34:14.366251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:43799 (local address 127.8.18.195:39683)
0810 11:34:14.366793 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0810 11:34:14.366888 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.254:43799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":426,"thread_start_us":286,"threads_started":1}
I20260810 11:34:14.842002 24153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:34:14.841000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:43799 (local address 127.8.18.194:52261)
0810 11:34:14.841649 (+ 649us) negotiation.cc:107] Waiting for socket to connect
0810 11:34:14.841796 (+ 147us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.254:43799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":527,"thread_start_us":289,"threads_started":1}
I20260810 11:34:15.306978 24154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:34:15.306162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:43799 (local address 127.8.18.193:60749)
0810 11:34:15.306711 (+ 549us) negotiation.cc:107] Waiting for socket to connect
0810 11:34:15.306808 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.254:43799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":424,"thread_start_us":283,"threads_started":1}
I20260810 11:34:15.368845 24155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:34:15.368029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:43799 (local address 127.8.18.195:59099)
0810 11:34:15.368539 (+ 510us) negotiation.cc:107] Waiting for socket to connect
0810 11:34:15.368654 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.254:43799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":401,"thread_start_us":264,"threads_started":1}
I20260810 11:34:15.844028 24156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:34:15.843148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:43799 (local address 127.8.18.194:55355)
0810 11:34:15.843721 (+ 573us) negotiation.cc:107] Waiting for socket to connect
0810 11:34:15.843830 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.254:43799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":450,"thread_start_us":302,"threads_started":1}
I20260810 11:34:16.309147 24157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:34:16.308233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:43799 (local address 127.8.18.193:59511)
0810 11:34:16.308811 (+ 578us) negotiation.cc:107] Waiting for socket to connect
0810 11:34:16.308937 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.254:43799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":451,"thread_start_us":294,"threads_started":1}
I20260810 11:34:16.370911 24158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:34:16.370011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:43799 (local address 127.8.18.195:52763)
0810 11:34:16.370635 (+ 624us) negotiation.cc:107] Waiting for socket to connect
0810 11:34:16.370738 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.254:43799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":507,"thread_start_us":359,"threads_started":1}
I20260810 11:34:16.846230 24159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:34:16.845223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:43799 (local address 127.8.18.194:60169)
0810 11:34:16.845849 (+ 626us) negotiation.cc:107] Waiting for socket to connect
0810 11:34:16.845972 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.254:43799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":514,"thread_start_us":343,"threads_started":1}
I20260810 11:34:17.311578 24160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:34:17.310491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:43799 (local address 127.8.18.193:59741)
0810 11:34:17.311189 (+ 698us) negotiation.cc:107] Waiting for socket to connect
0810 11:34:17.311296 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.254:43799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":525,"thread_start_us":325,"threads_started":1}
I20260810 11:34:17.373220 24161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:34:17.372217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:43799 (local address 127.8.18.195:33419)
0810 11:34:17.372780 (+ 563us) negotiation.cc:107] Waiting for socket to connect
0810 11:34:17.372957 (+ 177us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.254:43799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":426,"thread_start_us":291,"threads_started":1}
I20260810 11:34:17.848594 24162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:34:17.847608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:43799 (local address 127.8.18.194:59759)
0810 11:34:17.848229 (+ 621us) negotiation.cc:107] Waiting for socket to connect
0810 11:34:17.848325 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.254:43799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":471,"thread_start_us":321,"threads_started":1}
I20260810 11:34:18.314026 24163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:34:18.312952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:43799 (local address 127.8.18.193:35405)
0810 11:34:18.313719 (+ 767us) negotiation.cc:107] Waiting for socket to connect
0810 11:34:18.313828 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.254:43799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":587,"thread_start_us":418,"threads_started":1}
I20260810 11:34:18.375425 24164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:34:18.374365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:43799 (local address 127.8.18.195:58693)
0810 11:34:18.375019 (+ 654us) negotiation.cc:107] Waiting for socket to connect
0810 11:34:18.375159 (+ 140us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.254:43799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":521,"thread_start_us":391,"threads_started":1}
I20260810 11:34:18.850840 24166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:34:18.849990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:43799 (local address 127.8.18.194:41629)
0810 11:34:18.850538 (+ 548us) negotiation.cc:107] Waiting for socket to connect
0810 11:34:18.850639 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.254:43799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":441,"thread_start_us":271,"threads_started":1}
I20260810 11:34:19.316252 24167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:34:19.315357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:43799 (local address 127.8.18.193:42849)
0810 11:34:19.315949 (+ 592us) negotiation.cc:107] Waiting for socket to connect
0810 11:34:19.316054 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.254:43799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":455,"thread_start_us":296,"threads_started":1}
I20260810 11:34:19.347553 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 23682
I20260810 11:34:19.371670 24168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:34:19.349087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35768 (local address 127.8.18.193:41937)
0810 11:34:19.350087 (+ 1000us) server_negotiation.cc:206] Beginning negotiation
0810 11:34:19.350116 (+ 29us) server_negotiation.cc:399] Waiting for connection header
0810 11:34:19.350172 (+ 56us) server_negotiation.cc:407] Connection header received
0810 11:34:19.352558 (+ 2386us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:34:19.352571 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:34:19.353113 (+ 542us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:34:19.353729 (+ 616us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:34:19.355666 (+ 1937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:34:19.357128 (+ 1462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:34:19.358785 (+ 1657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:34:19.359819 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:34:19.360105 (+ 286us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:34:19.363553 (+ 3448us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:34:19.363563 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:34:19.363581 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:34:19.363634 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:34:19.367303 (+ 3669us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:34:19.368320 (+ 1017us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:34:19.368329 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:34:19.368337 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:34:19.368580 (+ 243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:34:19.369095 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:34:19.369103 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:34:19.369110 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:34:19.369864 (+ 754us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:34:19.370276 (+ 412us) server_negotiation.cc:1035] Waiting for connection context
0810 11:34:19.370800 (+ 524us) server_negotiation.cc:299] Negotiation successful
0810 11:34:19.371333 (+ 533us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":752,"thread_start_us":553,"threads_started":1}
I20260810 11:34:19.373441 23843 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:34:19.379976 24170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:34:19.376953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:43799 (local address 127.8.18.195:38649)
0810 11:34:19.379616 (+ 2663us) negotiation.cc:107] Waiting for socket to connect
0810 11:34:19.379765 (+ 149us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.254:43799: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2508,"thread_start_us":2337,"threads_started":1}
I20260810 11:34:19.467710 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 23682
I20260810 11:34:19.490186 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 23851
I20260810 11:34:19.511593 24171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:34:19.491343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60198 (local address 127.8.18.194:44985)
0810 11:34:19.491941 (+ 598us) server_negotiation.cc:206] Beginning negotiation
0810 11:34:19.491956 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 11:34:19.491995 (+ 39us) server_negotiation.cc:407] Connection header received
0810 11:34:19.493750 (+ 1755us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:34:19.493762 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:34:19.494226 (+ 464us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:34:19.494543 (+ 317us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:34:19.496187 (+ 1644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:34:19.497812 (+ 1625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:34:19.499218 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:34:19.499964 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:34:19.500132 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:34:19.503523 (+ 3391us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:34:19.503533 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:34:19.503554 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:34:19.503617 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:34:19.507334 (+ 3717us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:34:19.508308 (+ 974us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:34:19.508317 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:34:19.508325 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:34:19.508553 (+ 228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:34:19.509053 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:34:19.509057 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:34:19.509059 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:34:19.509811 (+ 752us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:34:19.510083 (+ 272us) server_negotiation.cc:1035] Waiting for connection context
0810 11:34:19.510757 (+ 674us) server_negotiation.cc:299] Negotiation successful
0810 11:34:19.511233 (+ 476us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":445,"thread_start_us":311,"threads_started":1}
I20260810 11:34:19.513638 23976 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:34:19.605803 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 23851
I20260810 11:34:19.627517 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 23984
I20260810 11:34:19.649765 24173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:34:19.628664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45202 (local address 127.8.18.195:43839)
0810 11:34:19.629362 (+ 698us) server_negotiation.cc:206] Beginning negotiation
0810 11:34:19.629378 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0810 11:34:19.629498 (+ 120us) server_negotiation.cc:407] Connection header received
0810 11:34:19.631297 (+ 1799us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:34:19.631310 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:34:19.631824 (+ 514us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:34:19.632271 (+ 447us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:34:19.634145 (+ 1874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:34:19.635568 (+ 1423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:34:19.637093 (+ 1525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:34:19.637985 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:34:19.638422 (+ 437us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:34:19.641830 (+ 3408us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:34:19.641838 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:34:19.641858 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:34:19.641910 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:34:19.645181 (+ 3271us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:34:19.646302 (+ 1121us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:34:19.646311 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:34:19.646319 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:34:19.646527 (+ 208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:34:19.647032 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:34:19.647039 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:34:19.647043 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:34:19.647855 (+ 812us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:34:19.648230 (+ 375us) server_negotiation.cc:1035] Waiting for connection context
0810 11:34:19.648800 (+ 570us) server_negotiation.cc:299] Negotiation successful
0810 11:34:19.649314 (+ 514us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":500,"thread_start_us":322,"threads_started":1}
I20260810 11:34:19.651468 24113 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:34:19.741560 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 23984
2026-08-10T11:34:19Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (25649 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 11:34:19 dist-test-slave-srgv krb5kdc[24177](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 11:34:19 dist-test-slave-srgv krb5kdc[24177](info): set up 2 sockets
Aug 10 11:34:19 dist-test-slave-srgv krb5kdc[24177](info): commencing operation
krb5kdc: starting...
W20260810 11:34:20.919679 8267 mini_kdc.cc:121] Time spent starting KDC: real 1.133s user 0.005s sys 0.035s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/krb5kdc/test-user.keytab.
Aug 10 11:34:21 dist-test-slave-srgv krb5kdc[24177](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361661, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T11:34:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T11:34:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.18.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.18.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.18.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.254.keytab.
Entry for principal kudu/127.8.18.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.254.keytab.
I20260810 11:34:21.251664 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.18.254:42587
--webserver_interface=127.8.18.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.254.keytab
--principal=kudu/127.8.18.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.18.212:33987
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.18.254:42587
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:34:21.651885 24190 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:34:21.652278 24190 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:34:21.652428 24190 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:34:21.660725 24190 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260810 11:34:21.660864 24190 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260810 11:34:21.660905 24190 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260810 11:34:21.660952 24190 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 11:34:21.661001 24190 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:34:21.661072 24190 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:34:21.661120 24190 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 11:34:21.661170 24190 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 11:34:21.672082 24190 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.18.212:33987
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.8.18.254:42587
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.254.keytab
--principal=kudu/127.8.18.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.18.254:42587
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.18.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:34:21.673609 24190 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:34:21.675122 24190 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:34:21.689335 24195 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:34:21.689399 24198 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:34:21.689360 24196 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:34:21.690630 24190 server_base.cc:1034] running on GCE node
I20260810 11:34:21.691593 24190 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:34:21.693224 24190 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:34:21.694520 24190 hybrid_clock.cc:648] HybridClock initialized: now 1786361661694447 us; error 67 us; skew 500 ppm
I20260810 11:34:21.694914 24190 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:34:21 dist-test-slave-srgv krb5kdc[24177](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361661, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:34:21.698694 24190 init.cc:377] Logged in from keytab as kudu/127.8.18.254@KRBTEST.COM (short username kudu)
I20260810 11:34:21.700519 24190 webserver.cc:460] Webserver started at http://127.8.18.254:39739/ using document root <none> and password file <none>
I20260810 11:34:21.701375 24190 fs_manager.cc:359] Metadata directory not provided
I20260810 11:34:21.701579 24190 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:34:21.701973 24190 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:34:21.706694 24190 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/master-0/data/instance:
uuid: "25ef7a60224c4f7bb5147d7c5ed8251c"
format_stamp: "Formatted at 2026-08-10 11:34:21 on dist-test-slave-srgv"
server_key: "59e635f02b278030db5011de502297ee"
server_key_iv: "b7d235b3fe008091868335d912f66813"
server_key_version: "encryptionkey@0"
I20260810 11:34:21.708200 24190 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/master-0/wal/instance:
uuid: "25ef7a60224c4f7bb5147d7c5ed8251c"
format_stamp: "Formatted at 2026-08-10 11:34:21 on dist-test-slave-srgv"
server_key: "59e635f02b278030db5011de502297ee"
server_key_iv: "b7d235b3fe008091868335d912f66813"
server_key_version: "encryptionkey@0"
I20260810 11:34:21.717115 24190 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.010s sys 0.001s
I20260810 11:34:21.721750 24205 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:34:21.723451 24190 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260810 11:34:21.723686 24190 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/master-0/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/master-0/wal
uuid: "25ef7a60224c4f7bb5147d7c5ed8251c"
format_stamp: "Formatted at 2026-08-10 11:34:21 on dist-test-slave-srgv"
server_key: "59e635f02b278030db5011de502297ee"
server_key_iv: "b7d235b3fe008091868335d912f66813"
server_key_version: "encryptionkey@0"
I20260810 11:34:21.724048 24190 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:34:21.740770 24190 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:34:21.746058 24190 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:34:21.746413 24190 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:34:21.768931 24190 rpc_server.cc:307] RPC server started. Bound to: 127.8.18.254:42587
I20260810 11:34:21.768952 24257 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.18.254:42587 every 8 connection(s)
I20260810 11:34:21.770548 24190 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
I20260810 11:34:21.775228 24258 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:34:21.777268 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 24190
I20260810 11:34:21.777559 8267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/master-0/wal/instance
I20260810 11:34:21.778539 8267 external_mini_cluster.cc:1428] Setting key 73cc1fda010daa1af17a3bf47a08bdc4
I20260810 11:34:21.791178 24258 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 25ef7a60224c4f7bb5147d7c5ed8251c: Bootstrap starting.
I20260810 11:34:21.795948 24258 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 25ef7a60224c4f7bb5147d7c5ed8251c: Neither blocks nor log segments found. Creating new log.
Aug 10 11:34:21 dist-test-slave-srgv krb5kdc[24177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361661, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.18.254@KRBTEST.COM
I20260810 11:34:21.797570 24258 log.cc:824] T 00000000000000000000000000000000 P 25ef7a60224c4f7bb5147d7c5ed8251c: Log is configured to *not* fsync() on all Append() calls
I20260810 11:34:21.802502 24258 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 25ef7a60224c4f7bb5147d7c5ed8251c: No bootstrap required, opened a new log
I20260810 11:34:21.810982 24261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:34:21.781756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55436 (local address 127.8.18.254:42587)
0810 11:34:21.782944 (+ 1188us) server_negotiation.cc:206] Beginning negotiation
0810 11:34:21.782958 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 11:34:21.783008 (+ 50us) server_negotiation.cc:407] Connection header received
0810 11:34:21.784977 (+ 1969us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:34:21.785006 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:34:21.785614 (+ 608us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:34:21.786145 (+ 531us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:34:21.788473 (+ 2328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:34:21.790086 (+ 1613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:34:21.792188 (+ 2102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:34:21.792965 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:34:21.793206 (+ 241us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:34:21.799493 (+ 6287us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:34:21.799508 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:34:21.799539 (+ 31us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:34:21.799598 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:34:21.804496 (+ 4898us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:34:21.805730 (+ 1234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:34:21.805741 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:34:21.805752 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:34:21.806031 (+ 279us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:34:21.806688 (+ 657us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:34:21.806693 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:34:21.806697 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:34:21.807679 (+ 982us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:34:21.808069 (+ 390us) server_negotiation.cc:1035] Waiting for connection context
0810 11:34:21.808648 (+ 579us) server_negotiation.cc:299] Negotiation successful
0810 11:34:21.809290 (+ 642us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":905,"thread_start_us":557,"threads_started":1}
I20260810 11:34:21.814540 24258 raft_consensus.cc:348] T 00000000000000000000000000000000 P 25ef7a60224c4f7bb5147d7c5ed8251c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25ef7a60224c4f7bb5147d7c5ed8251c" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 42587 } }
I20260810 11:34:21.815101 24258 raft_consensus.cc:374] T 00000000000000000000000000000000 P 25ef7a60224c4f7bb5147d7c5ed8251c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:34:21.815229 24258 raft_consensus.cc:729] T 00000000000000000000000000000000 P 25ef7a60224c4f7bb5147d7c5ed8251c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25ef7a60224c4f7bb5147d7c5ed8251c, State: Initialized, Role: FOLLOWER
I20260810 11:34:21.815964 24258 consensus_queue.cc:260] T 00000000000000000000000000000000 P 25ef7a60224c4f7bb5147d7c5ed8251c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25ef7a60224c4f7bb5147d7c5ed8251c" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 42587 } }
I20260810 11:34:21.816231 24258 raft_consensus.cc:388] T 00000000000000000000000000000000 P 25ef7a60224c4f7bb5147d7c5ed8251c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 11:34:21.816412 24258 raft_consensus.cc:482] T 00000000000000000000000000000000 P 25ef7a60224c4f7bb5147d7c5ed8251c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 11:34:21.816638 24258 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 25ef7a60224c4f7bb5147d7c5ed8251c [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:34:21.819265 24258 raft_consensus.cc:504] T 00000000000000000000000000000000 P 25ef7a60224c4f7bb5147d7c5ed8251c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25ef7a60224c4f7bb5147d7c5ed8251c" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 42587 } }
I20260810 11:34:21.820024 24258 leader_election.cc:302] T 00000000000000000000000000000000 P 25ef7a60224c4f7bb5147d7c5ed8251c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 25ef7a60224c4f7bb5147d7c5ed8251c; no voters:
I20260810 11:34:21.820892 24258 leader_election.cc:288] T 00000000000000000000000000000000 P 25ef7a60224c4f7bb5147d7c5ed8251c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 11:34:21.821048 24263 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 25ef7a60224c4f7bb5147d7c5ed8251c [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:34:21.822187 24263 raft_consensus.cc:686] T 00000000000000000000000000000000 P 25ef7a60224c4f7bb5147d7c5ed8251c [term 1 LEADER]: Becoming Leader. State: Replica: 25ef7a60224c4f7bb5147d7c5ed8251c, State: Running, Role: LEADER
I20260810 11:34:21.822860 24263 consensus_queue.cc:237] T 00000000000000000000000000000000 P 25ef7a60224c4f7bb5147d7c5ed8251c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25ef7a60224c4f7bb5147d7c5ed8251c" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 42587 } }
I20260810 11:34:21.823753 24258 sys_catalog.cc:564] T 00000000000000000000000000000000 P 25ef7a60224c4f7bb5147d7c5ed8251c [sys.catalog]: configured and running, proceeding with master startup.
I20260810 11:34:21.826992 24265 sys_catalog.cc:455] T 00000000000000000000000000000000 P 25ef7a60224c4f7bb5147d7c5ed8251c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 25ef7a60224c4f7bb5147d7c5ed8251c. Latest consensus state: current_term: 1 leader_uuid: "25ef7a60224c4f7bb5147d7c5ed8251c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25ef7a60224c4f7bb5147d7c5ed8251c" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 42587 } } }
I20260810 11:34:21.827018 24264 sys_catalog.cc:455] T 00000000000000000000000000000000 P 25ef7a60224c4f7bb5147d7c5ed8251c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "25ef7a60224c4f7bb5147d7c5ed8251c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25ef7a60224c4f7bb5147d7c5ed8251c" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 42587 } } }
I20260810 11:34:21.827517 24265 sys_catalog.cc:458] T 00000000000000000000000000000000 P 25ef7a60224c4f7bb5147d7c5ed8251c [sys.catalog]: This master's current role is: LEADER
I20260810 11:34:21.827524 24264 sys_catalog.cc:458] T 00000000000000000000000000000000 P 25ef7a60224c4f7bb5147d7c5ed8251c [sys.catalog]: This master's current role is: LEADER
I20260810 11:34:21.831076 24271 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 11:34:21.837688 24271 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 11:34:21.848548 24271 catalog_manager.cc:1347] Generated new cluster ID: e2a8e20598364125b81bb3c2699a96d8
I20260810 11:34:21.848727 24271 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 11:34:21.859611 24271 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 11:34:21.861843 24271 catalog_manager.cc:1504] Loading token signing keys...
I20260810 11:34:21.871997 24271 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 25ef7a60224c4f7bb5147d7c5ed8251c: Generated new TSK 0
I20260810 11:34:21.873121 24271 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.18.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.18.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.18.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.193.keytab.
Entry for principal kudu/127.8.18.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.193.keytab.
I20260810 11:34:21.954748 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.18.193:0
--local_ip_for_outbound_sockets=127.8.18.193
--webserver_interface=127.8.18.193
--webserver_port=0
--tserver_master_addrs=127.8.18.254:42587
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.193.keytab
--principal=kudu/127.8.18.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.18.212:33987
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:34:22.374308 24285 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:34:22.374696 24285 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:34:22.374881 24285 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:34:22.383237 24285 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:34:22.383396 24285 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:34:22.383597 24285 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.18.193
I20260810 11:34:22.395170 24285 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.18.212:33987
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.193.keytab
--principal=kudu/127.8.18.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.18.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.18.193
--webserver_port=0
--tserver_master_addrs=127.8.18.254:42587
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.18.193
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:34:22.396967 24285 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:34:22.398610 24285 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:34:22.414709 24291 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:34:22.414606 24293 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:34:22.414757 24290 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:34:22.416097 24285 server_base.cc:1034] running on GCE node
I20260810 11:34:22.417001 24285 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:34:22.418279 24285 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:34:22.419541 24285 hybrid_clock.cc:648] HybridClock initialized: now 1786361662419490 us; error 50 us; skew 500 ppm
I20260810 11:34:22.419991 24285 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:34:22 dist-test-slave-srgv krb5kdc[24177](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361662, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:34:22.424141 24285 init.cc:377] Logged in from keytab as kudu/127.8.18.193@KRBTEST.COM (short username kudu)
I20260810 11:34:22.426329 24285 webserver.cc:460] Webserver started at http://127.8.18.193:38413/ using document root <none> and password file <none>
I20260810 11:34:22.427187 24285 fs_manager.cc:359] Metadata directory not provided
I20260810 11:34:22.427386 24285 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:34:22.427771 24285 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:34:22.431774 24285 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/ts-0/data/instance:
uuid: "8a82dd1f6f654360b68d7ff014fe5e71"
format_stamp: "Formatted at 2026-08-10 11:34:22 on dist-test-slave-srgv"
server_key: "d17b1c80112eb8b90b6e989b0c672614"
server_key_iv: "bb6e0401ef0735452bbcac4e2c4c779f"
server_key_version: "encryptionkey@0"
I20260810 11:34:22.432870 24285 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/ts-0/wal/instance:
uuid: "8a82dd1f6f654360b68d7ff014fe5e71"
format_stamp: "Formatted at 2026-08-10 11:34:22 on dist-test-slave-srgv"
server_key: "d17b1c80112eb8b90b6e989b0c672614"
server_key_iv: "bb6e0401ef0735452bbcac4e2c4c779f"
server_key_version: "encryptionkey@0"
I20260810 11:34:22.439718 24285 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.009s sys 0.000s
I20260810 11:34:22.444466 24300 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:34:22.446656 24285 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260810 11:34:22.446910 24285 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/ts-0/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/ts-0/wal
uuid: "8a82dd1f6f654360b68d7ff014fe5e71"
format_stamp: "Formatted at 2026-08-10 11:34:22 on dist-test-slave-srgv"
server_key: "d17b1c80112eb8b90b6e989b0c672614"
server_key_iv: "bb6e0401ef0735452bbcac4e2c4c779f"
server_key_version: "encryptionkey@0"
I20260810 11:34:22.447177 24285 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:34:22.479671 24285 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:34:22.485620 24285 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:34:22.485986 24285 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:34:22.487169 24285 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:34:22.488858 24285 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:34:22.488999 24285 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:34:22.489171 24285 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:34:22.489267 24285 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:34:22.529536 24285 rpc_server.cc:307] RPC server started. Bound to: 127.8.18.193:43511
I20260810 11:34:22.529647 24415 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.18.193:43511 every 8 connection(s)
I20260810 11:34:22.531198 24285 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/ts-0/data/info.pb
I20260810 11:34:22.535216 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 24285
I20260810 11:34:22.535495 8267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/ts-0/wal/instance
I20260810 11:34:22.536293 8267 external_mini_cluster.cc:1428] Setting key fb5136aa3b0492932144b2b1264d0c3e
Aug 10 11:34:22 dist-test-slave-srgv krb5kdc[24177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361662, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.193@KRBTEST.COM for kudu/127.8.18.254@KRBTEST.COM
I20260810 11:34:22.557570 24418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:34:22.534349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.193:60503 (local address 127.8.18.254:42587)
0810 11:34:22.534953 (+ 604us) server_negotiation.cc:206] Beginning negotiation
0810 11:34:22.534960 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:34:22.536102 (+ 1142us) server_negotiation.cc:407] Connection header received
0810 11:34:22.538218 (+ 2116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:34:22.538224 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:34:22.538316 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:34:22.538409 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:34:22.540249 (+ 1840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:34:22.541526 (+ 1277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:34:22.543445 (+ 1919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:34:22.544222 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:34:22.544392 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:34:22.548953 (+ 4561us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:34:22.548963 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:34:22.548971 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:34:22.549027 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:34:22.552592 (+ 3565us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:34:22.554147 (+ 1555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:34:22.554153 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:34:22.554156 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:34:22.554354 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:34:22.555132 (+ 778us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:34:22.555136 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:34:22.555139 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:34:22.555521 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:34:22.555724 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0810 11:34:22.556862 (+ 1138us) server_negotiation.cc:299] Negotiation successful
0810 11:34:22.557105 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":464,"thread_start_us":265,"threads_started":1}
I20260810 11:34:22.559201 24419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:34:22.534961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:42587 (local address 127.8.18.193:60503)
0810 11:34:22.535880 (+ 919us) negotiation.cc:107] Waiting for socket to connect
0810 11:34:22.535944 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0810 11:34:22.537922 (+ 1978us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:34:22.538743 (+ 821us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:34:22.538753 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:34:22.539338 (+ 585us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:34:22.540003 (+ 665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:34:22.540023 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:34:22.541750 (+ 1727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:34:22.541756 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:34:22.543230 (+ 1474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:34:22.543248 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:34:22.544535 (+ 1287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:34:22.544541 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:34:22.544779 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:34:22.545675 (+ 896us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:34:22.545702 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:34:22.548604 (+ 2902us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:34:22.552825 (+ 4221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:34:22.552840 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:34:22.552863 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:34:22.553886 (+ 1023us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:34:22.554569 (+ 683us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:34:22.554578 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:34:22.554584 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:34:22.554932 (+ 348us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:34:22.555738 (+ 806us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:34:22.555749 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:34:22.556334 (+ 585us) client_negotiation.cc:769] Sending connection context
0810 11:34:22.556690 (+ 356us) client_negotiation.cc:240] Negotiation successful
0810 11:34:22.557392 (+ 702us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":611,"thread_start_us":394,"threads_started":1}
I20260810 11:34:22.561486 24416 heartbeater.cc:347] Connected to a master server at 127.8.18.254:42587
I20260810 11:34:22.561894 24416 heartbeater.cc:464] Registering TS with master...
I20260810 11:34:22.562806 24416 heartbeater.cc:511] Master 127.8.18.254:42587 requested a full tablet report, sending...
I20260810 11:34:22.565577 24223 ts_manager.cc:194] Registered new tserver with Master: 8a82dd1f6f654360b68d7ff014fe5e71 (127.8.18.193:43511)
I20260810 11:34:22.567291 24223 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.18.193@KRBTEST.COM'} at 127.8.18.193:60503
WARNING: no policy specified for kudu/127.8.18.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.18.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.18.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.194.keytab.
Entry for principal kudu/127.8.18.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.194.keytab.
I20260810 11:34:22.613837 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.18.194:0
--local_ip_for_outbound_sockets=127.8.18.194
--webserver_interface=127.8.18.194
--webserver_port=0
--tserver_master_addrs=127.8.18.254:42587
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.194.keytab
--principal=kudu/127.8.18.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.18.212:33987
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:34:23.012032 24423 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:34:23.012449 24423 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:34:23.012622 24423 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:34:23.020746 24423 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:34:23.020967 24423 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:34:23.021168 24423 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.18.194
I20260810 11:34:23.032958 24423 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.18.212:33987
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.194.keytab
--principal=kudu/127.8.18.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.18.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.18.194
--webserver_port=0
--tserver_master_addrs=127.8.18.254:42587
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.18.194
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:34:23.034505 24423 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:34:23.035982 24423 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:34:23.051307 24428 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:34:23.051405 24429 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:34:23.051353 24431 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:34:23.052556 24423 server_base.cc:1034] running on GCE node
I20260810 11:34:23.053318 24423 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:34:23.054522 24423 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:34:23.055790 24423 hybrid_clock.cc:648] HybridClock initialized: now 1786361663055724 us; error 54 us; skew 500 ppm
I20260810 11:34:23.056265 24423 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:34:23 dist-test-slave-srgv krb5kdc[24177](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361663, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:34:23.064041 24423 init.cc:377] Logged in from keytab as kudu/127.8.18.194@KRBTEST.COM (short username kudu)
I20260810 11:34:23.066093 24423 webserver.cc:460] Webserver started at http://127.8.18.194:44741/ using document root <none> and password file <none>
I20260810 11:34:23.066902 24423 fs_manager.cc:359] Metadata directory not provided
I20260810 11:34:23.067047 24423 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:34:23.067368 24423 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:34:23.070863 24423 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/ts-1/data/instance:
uuid: "1103ed0eb8974fd8bc3eb2955eacbc78"
format_stamp: "Formatted at 2026-08-10 11:34:23 on dist-test-slave-srgv"
server_key: "bddc7322a41df9efaed932734d963059"
server_key_iv: "1fa90db762df1c07e2bda8f2688f461b"
server_key_version: "encryptionkey@0"
I20260810 11:34:23.071749 24423 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/ts-1/wal/instance:
uuid: "1103ed0eb8974fd8bc3eb2955eacbc78"
format_stamp: "Formatted at 2026-08-10 11:34:23 on dist-test-slave-srgv"
server_key: "bddc7322a41df9efaed932734d963059"
server_key_iv: "1fa90db762df1c07e2bda8f2688f461b"
server_key_version: "encryptionkey@0"
I20260810 11:34:23.078366 24423 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260810 11:34:23.082772 24438 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:34:23.084659 24423 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260810 11:34:23.084926 24423 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/ts-1/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/ts-1/wal
uuid: "1103ed0eb8974fd8bc3eb2955eacbc78"
format_stamp: "Formatted at 2026-08-10 11:34:23 on dist-test-slave-srgv"
server_key: "bddc7322a41df9efaed932734d963059"
server_key_iv: "1fa90db762df1c07e2bda8f2688f461b"
server_key_version: "encryptionkey@0"
I20260810 11:34:23.085201 24423 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:34:23.115293 24423 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:34:23.121966 24423 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:34:23.122296 24423 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:34:23.123407 24423 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:34:23.125291 24423 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:34:23.125415 24423 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:34:23.125638 24423 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:34:23.125728 24423 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:34:23.164424 24423 rpc_server.cc:307] RPC server started. Bound to: 127.8.18.194:34511
I20260810 11:34:23.164476 24553 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.18.194:34511 every 8 connection(s)
I20260810 11:34:23.165978 24423 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/ts-1/data/info.pb
I20260810 11:34:23.169457 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 24423
I20260810 11:34:23.169762 8267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/ts-1/wal/instance
I20260810 11:34:23.170660 8267 external_mini_cluster.cc:1428] Setting key 97f659088e37d3c584f3185967bc1a73
Aug 10 11:34:23 dist-test-slave-srgv krb5kdc[24177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361663, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.194@KRBTEST.COM for kudu/127.8.18.254@KRBTEST.COM
I20260810 11:34:23.195078 24556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:34:23.169506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.194:46793 (local address 127.8.18.254:42587)
0810 11:34:23.172355 (+ 2849us) server_negotiation.cc:206] Beginning negotiation
0810 11:34:23.172361 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:34:23.172382 (+ 21us) server_negotiation.cc:407] Connection header received
0810 11:34:23.174000 (+ 1618us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:34:23.174004 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:34:23.174082 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:34:23.174196 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:34:23.176639 (+ 2443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:34:23.177505 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:34:23.180271 (+ 2766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:34:23.180801 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:34:23.180928 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:34:23.186307 (+ 5379us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:34:23.186315 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:34:23.186321 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:34:23.186384 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:34:23.190822 (+ 4438us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:34:23.191984 (+ 1162us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:34:23.191993 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:34:23.191999 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:34:23.192193 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:34:23.192694 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:34:23.192702 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:34:23.192708 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:34:23.193190 (+ 482us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:34:23.193363 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0810 11:34:23.194248 (+ 885us) server_negotiation.cc:299] Negotiation successful
0810 11:34:23.194562 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2736,"thread_start_us":2572,"threads_started":1}
I20260810 11:34:23.195899 24557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:34:23.170113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:42587 (local address 127.8.18.194:46793)
0810 11:34:23.171130 (+ 1017us) negotiation.cc:107] Waiting for socket to connect
0810 11:34:23.171184 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0810 11:34:23.173630 (+ 2446us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:34:23.174519 (+ 889us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:34:23.174534 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:34:23.175429 (+ 895us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:34:23.176392 (+ 963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:34:23.176425 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:34:23.177698 (+ 1273us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:34:23.177706 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:34:23.180033 (+ 2327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:34:23.180060 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:34:23.181095 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:34:23.181103 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:34:23.181520 (+ 417us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:34:23.182393 (+ 873us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:34:23.182430 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:34:23.185916 (+ 3486us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:34:23.191214 (+ 5298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:34:23.191223 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:34:23.191238 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:34:23.191812 (+ 574us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:34:23.192323 (+ 511us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:34:23.192327 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:34:23.192329 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:34:23.192569 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:34:23.193338 (+ 769us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:34:23.193346 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:34:23.193821 (+ 475us) client_negotiation.cc:769] Sending connection context
0810 11:34:23.194135 (+ 314us) client_negotiation.cc:240] Negotiation successful
0810 11:34:23.194613 (+ 478us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":718,"thread_start_us":448,"threads_started":1}
I20260810 11:34:23.197772 24554 heartbeater.cc:347] Connected to a master server at 127.8.18.254:42587
I20260810 11:34:23.198201 24554 heartbeater.cc:464] Registering TS with master...
I20260810 11:34:23.199268 24554 heartbeater.cc:511] Master 127.8.18.254:42587 requested a full tablet report, sending...
I20260810 11:34:23.201315 24223 ts_manager.cc:194] Registered new tserver with Master: 1103ed0eb8974fd8bc3eb2955eacbc78 (127.8.18.194:34511)
I20260810 11:34:23.202288 24223 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.18.194@KRBTEST.COM'} at 127.8.18.194:46793
WARNING: no policy specified for kudu/127.8.18.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.18.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.18.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.195.keytab.
Entry for principal kudu/127.8.18.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.195.keytab.
I20260810 11:34:23.247964 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.18.195:0
--local_ip_for_outbound_sockets=127.8.18.195
--webserver_interface=127.8.18.195
--webserver_port=0
--tserver_master_addrs=127.8.18.254:42587
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.195.keytab
--principal=kudu/127.8.18.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.18.212:33987
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 11:34:23.571447 24416 heartbeater.cc:503] Master 127.8.18.254:42587 was elected leader, sending a full tablet report...
W20260810 11:34:23.632632 24560 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:34:23.632966 24560 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:34:23.633134 24560 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:34:23.641552 24560 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:34:23.641705 24560 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:34:23.641922 24560 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.18.195
I20260810 11:34:23.653689 24560 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.18.212:33987
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.195.keytab
--principal=kudu/127.8.18.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.18.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.18.195
--webserver_port=0
--tserver_master_addrs=127.8.18.254:42587
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.18.195
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:34:23.655231 24560 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:34:23.656721 24560 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:34:23.673233 24566 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:34:23.673177 24568 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:34:23.673213 24565 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:34:23.674362 24560 server_base.cc:1034] running on GCE node
I20260810 11:34:23.675180 24560 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:34:23.676411 24560 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:34:23.677685 24560 hybrid_clock.cc:648] HybridClock initialized: now 1786361663677635 us; error 52 us; skew 500 ppm
I20260810 11:34:23.678076 24560 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:34:23 dist-test-slave-srgv krb5kdc[24177](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361663, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:34:23.681910 24560 init.cc:377] Logged in from keytab as kudu/127.8.18.195@KRBTEST.COM (short username kudu)
I20260810 11:34:23.683691 24560 webserver.cc:460] Webserver started at http://127.8.18.195:46223/ using document root <none> and password file <none>
I20260810 11:34:23.684487 24560 fs_manager.cc:359] Metadata directory not provided
I20260810 11:34:23.684613 24560 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:34:23.684918 24560 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:34:23.688410 24560 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/ts-2/data/instance:
uuid: "f87e487a91b542408b4a24ffc52421be"
format_stamp: "Formatted at 2026-08-10 11:34:23 on dist-test-slave-srgv"
server_key: "f937034459273ddb79f12ef371db5203"
server_key_iv: "c5654e1e7bfbd521d2210a671cc5a52a"
server_key_version: "encryptionkey@0"
I20260810 11:34:23.689352 24560 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/ts-2/wal/instance:
uuid: "f87e487a91b542408b4a24ffc52421be"
format_stamp: "Formatted at 2026-08-10 11:34:23 on dist-test-slave-srgv"
server_key: "f937034459273ddb79f12ef371db5203"
server_key_iv: "c5654e1e7bfbd521d2210a671cc5a52a"
server_key_version: "encryptionkey@0"
I20260810 11:34:23.696026 24560 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260810 11:34:23.700412 24575 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:34:23.702467 24560 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.001s
I20260810 11:34:23.702682 24560 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/ts-2/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/ts-2/wal
uuid: "f87e487a91b542408b4a24ffc52421be"
format_stamp: "Formatted at 2026-08-10 11:34:23 on dist-test-slave-srgv"
server_key: "f937034459273ddb79f12ef371db5203"
server_key_iv: "c5654e1e7bfbd521d2210a671cc5a52a"
server_key_version: "encryptionkey@0"
I20260810 11:34:23.702895 24560 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:34:23.737372 24560 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:34:23.742635 24560 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:34:23.743000 24560 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:34:23.744201 24560 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:34:23.746333 24560 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:34:23.746429 24560 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:34:23.746595 24560 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:34:23.746670 24560 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:34:23.785445 24560 rpc_server.cc:307] RPC server started. Bound to: 127.8.18.195:40573
I20260810 11:34:23.785552 24689 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.18.195:40573 every 8 connection(s)
I20260810 11:34:23.787096 24560 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/ts-2/data/info.pb
I20260810 11:34:23.792632 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 24560
I20260810 11:34:23.792868 8267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/ts-2/wal/instance
I20260810 11:34:23.793555 8267 external_mini_cluster.cc:1428] Setting key d31d296e730d17f153db04d95bf17829
Aug 10 11:34:23 dist-test-slave-srgv krb5kdc[24177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361663, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.195@KRBTEST.COM for kudu/127.8.18.254@KRBTEST.COM
I20260810 11:34:23.812131 24692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:34:23.789983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.195:54077 (local address 127.8.18.254:42587)
0810 11:34:23.790497 (+ 514us) server_negotiation.cc:206] Beginning negotiation
0810 11:34:23.790501 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 11:34:23.791748 (+ 1247us) server_negotiation.cc:407] Connection header received
0810 11:34:23.794744 (+ 2996us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:34:23.794751 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:34:23.794854 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:34:23.794981 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:34:23.797275 (+ 2294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:34:23.798174 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:34:23.799821 (+ 1647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:34:23.800229 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:34:23.800329 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:34:23.804415 (+ 4086us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:34:23.804421 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:34:23.804424 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:34:23.804476 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:34:23.807839 (+ 3363us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:34:23.809131 (+ 1292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:34:23.809138 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:34:23.809141 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:34:23.809364 (+ 223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:34:23.809934 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:34:23.809940 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:34:23.809944 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:34:23.810383 (+ 439us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:34:23.810558 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0810 11:34:23.811422 (+ 864us) server_negotiation.cc:299] Negotiation successful
0810 11:34:23.811718 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":407,"thread_start_us":281,"threads_started":1}
I20260810 11:34:23.813272 24693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:34:23.790611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:42587 (local address 127.8.18.195:54077)
0810 11:34:23.791432 (+ 821us) negotiation.cc:107] Waiting for socket to connect
0810 11:34:23.791488 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0810 11:34:23.794306 (+ 2818us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:34:23.795345 (+ 1039us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:34:23.795359 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:34:23.796259 (+ 900us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:34:23.797037 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:34:23.797086 (+ 49us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:34:23.798348 (+ 1262us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:34:23.798352 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:34:23.799664 (+ 1312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:34:23.799677 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:34:23.800434 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:34:23.800439 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:34:23.800656 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:34:23.801469 (+ 813us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:34:23.801505 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:34:23.804079 (+ 2574us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:34:23.808206 (+ 4127us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:34:23.808215 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:34:23.808231 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:34:23.808975 (+ 744us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:34:23.809540 (+ 565us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:34:23.809545 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:34:23.809549 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:34:23.809802 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:34:23.810543 (+ 741us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:34:23.810553 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:34:23.810972 (+ 419us) client_negotiation.cc:769] Sending connection context
0810 11:34:23.811306 (+ 334us) client_negotiation.cc:240] Negotiation successful
0810 11:34:23.811927 (+ 621us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":545,"thread_start_us":360,"threads_started":1}
I20260810 11:34:23.815063 24690 heartbeater.cc:347] Connected to a master server at 127.8.18.254:42587
I20260810 11:34:23.815413 24690 heartbeater.cc:464] Registering TS with master...
I20260810 11:34:23.816262 24690 heartbeater.cc:511] Master 127.8.18.254:42587 requested a full tablet report, sending...
I20260810 11:34:23.818266 24223 ts_manager.cc:194] Registered new tserver with Master: f87e487a91b542408b4a24ffc52421be (127.8.18.195:40573)
I20260810 11:34:23.819211 24223 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.18.195@KRBTEST.COM'} at 127.8.18.195:54077
I20260810 11:34:23.823215 8267 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 11:34:23.844094 24692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:34:23.828716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55438 (local address 127.8.18.254:42587)
0810 11:34:23.828922 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0810 11:34:23.828926 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 11:34:23.829568 (+ 642us) server_negotiation.cc:407] Connection header received
0810 11:34:23.829777 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:34:23.829781 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:34:23.829861 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:34:23.829971 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:34:23.831876 (+ 1905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:34:23.832872 (+ 996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:34:23.834595 (+ 1723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:34:23.835065 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:34:23.835215 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:34:23.838101 (+ 2886us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:34:23.838108 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:34:23.838112 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:34:23.838158 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:34:23.841279 (+ 3121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:34:23.842090 (+ 811us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:34:23.842095 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:34:23.842097 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:34:23.842277 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:34:23.842657 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:34:23.842660 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:34:23.842662 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:34:23.843008 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:34:23.843255 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0810 11:34:23.843624 (+ 369us) server_negotiation.cc:299] Negotiation successful
0810 11:34:23.843827 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260810 11:34:23.849030 24223 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55438:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 11:34:23.856709 24223 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 11:34:23.895030 24705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:34:23.872381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.194:34511 (local address 127.0.0.1:52980)
0810 11:34:23.884205 (+ 11824us) negotiation.cc:107] Waiting for socket to connect
0810 11:34:23.884243 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0810 11:34:23.884466 (+ 223us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:34:23.886337 (+ 1871us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:34:23.886346 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:34:23.886377 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:34:23.886787 (+ 410us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:34:23.886798 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:34:23.889135 (+ 2337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:34:23.889140 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:34:23.891956 (+ 2816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:34:23.891972 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:34:23.894070 (+ 2098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:34:23.894079 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:34:23.894329 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:34:23.894380 (+ 51us) client_negotiation.cc:769] Sending connection context
0810 11:34:23.894475 (+ 95us) client_negotiation.cc:240] Negotiation successful
0810 11:34:23.894646 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11591,"mutex_wait_us":6,"thread_start_us":13407,"threads_started":1}
I20260810 11:34:23.895195 24704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:34:23.871953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.195:40573 (local address 127.0.0.1:58648)
0810 11:34:23.885935 (+ 13982us) negotiation.cc:107] Waiting for socket to connect
0810 11:34:23.885964 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0810 11:34:23.886123 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:34:23.887465 (+ 1342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:34:23.887470 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:34:23.887502 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:34:23.887941 (+ 439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:34:23.887953 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:34:23.889968 (+ 2015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:34:23.889976 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:34:23.892014 (+ 2038us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:34:23.892036 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:34:23.894211 (+ 2175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:34:23.894219 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:34:23.894417 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:34:23.894457 (+ 40us) client_negotiation.cc:769] Sending connection context
0810 11:34:23.894529 (+ 72us) client_negotiation.cc:240] Negotiation successful
0810 11:34:23.894700 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13823,"mutex_wait_us":1956,"thread_start_us":8129,"threads_started":1}
I20260810 11:34:23.896123 24708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:34:23.873381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52980 (local address 127.8.18.194:34511)
0810 11:34:23.881838 (+ 8457us) server_negotiation.cc:206] Beginning negotiation
0810 11:34:23.881852 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 11:34:23.884678 (+ 2826us) server_negotiation.cc:407] Connection header received
0810 11:34:23.884844 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:34:23.884852 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:34:23.885192 (+ 340us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:34:23.885488 (+ 296us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:34:23.886987 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:34:23.888689 (+ 1702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:34:23.892170 (+ 3481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:34:23.893767 (+ 1597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:34:23.895359 (+ 1592us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:34:23.895624 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0810 11:34:23.895706 (+ 82us) server_negotiation.cc:299] Negotiation successful
0810 11:34:23.895857 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8253,"thread_start_us":8070,"threads_started":1}
I20260810 11:34:23.896706 24707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:34:23.873025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58648 (local address 127.8.18.195:40573)
0810 11:34:23.885160 (+ 12135us) server_negotiation.cc:206] Beginning negotiation
0810 11:34:23.885170 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 11:34:23.886135 (+ 965us) server_negotiation.cc:407] Connection header received
0810 11:34:23.886419 (+ 284us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:34:23.886431 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:34:23.886799 (+ 368us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:34:23.887088 (+ 289us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:34:23.888174 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:34:23.889723 (+ 1549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:34:23.892237 (+ 2514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:34:23.893921 (+ 1684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:34:23.895810 (+ 1889us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:34:23.896115 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0810 11:34:23.896207 (+ 92us) server_negotiation.cc:299] Negotiation successful
0810 11:34:23.896389 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11918,"thread_start_us":12164,"threads_started":1}
I20260810 11:34:23.897333 24709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:34:23.871944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.193:43511 (local address 127.0.0.1:59218)
0810 11:34:23.882528 (+ 10584us) negotiation.cc:107] Waiting for socket to connect
0810 11:34:23.882580 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0810 11:34:23.882815 (+ 235us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:34:23.888489 (+ 5674us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:34:23.888495 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:34:23.888524 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:34:23.888922 (+ 398us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:34:23.888938 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:34:23.891634 (+ 2696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:34:23.891640 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:34:23.894070 (+ 2430us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:34:23.894089 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:34:23.896500 (+ 2411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:34:23.896507 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:34:23.896690 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:34:23.896734 (+ 44us) client_negotiation.cc:769] Sending connection context
0810 11:34:23.896831 (+ 97us) client_negotiation.cc:240] Negotiation successful
0810 11:34:23.896973 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10301,"thread_start_us":13658,"threads_started":1}
I20260810 11:34:23.897521 24706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:34:23.872870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59218 (local address 127.8.18.193:43511)
0810 11:34:23.887534 (+ 14664us) server_negotiation.cc:206] Beginning negotiation
0810 11:34:23.887545 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 11:34:23.887578 (+ 33us) server_negotiation.cc:407] Connection header received
0810 11:34:23.887680 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:34:23.887688 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:34:23.887937 (+ 249us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:34:23.888165 (+ 228us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:34:23.889138 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:34:23.891411 (+ 2273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:34:23.894338 (+ 2927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:34:23.896044 (+ 1706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:34:23.896235 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:34:23.896443 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0810 11:34:23.897011 (+ 568us) server_negotiation.cc:299] Negotiation successful
0810 11:34:23.897163 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14453,"thread_start_us":14736,"threads_started":1}
I20260810 11:34:23.900614 24625 tablet_service.cc:1467] Processing CreateTablet for tablet c1152ff5766f4e2e90b97218a5723d20 (DEFAULT_TABLE table=test-table [id=59d3a3102c4c4d56a327d4e9e4efc0ba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:34:23.900648 24488 tablet_service.cc:1467] Processing CreateTablet for tablet c1152ff5766f4e2e90b97218a5723d20 (DEFAULT_TABLE table=test-table [id=59d3a3102c4c4d56a327d4e9e4efc0ba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:34:23.902096 24625 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c1152ff5766f4e2e90b97218a5723d20. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:34:23.902271 24488 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c1152ff5766f4e2e90b97218a5723d20. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:34:23.901804 24350 tablet_service.cc:1467] Processing CreateTablet for tablet c1152ff5766f4e2e90b97218a5723d20 (DEFAULT_TABLE table=test-table [id=59d3a3102c4c4d56a327d4e9e4efc0ba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:34:23.903395 24350 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c1152ff5766f4e2e90b97218a5723d20. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:34:23.916213 24710 tablet_bootstrap.cc:492] T c1152ff5766f4e2e90b97218a5723d20 P f87e487a91b542408b4a24ffc52421be: Bootstrap starting.
I20260810 11:34:23.919744 24711 tablet_bootstrap.cc:492] T c1152ff5766f4e2e90b97218a5723d20 P 1103ed0eb8974fd8bc3eb2955eacbc78: Bootstrap starting.
I20260810 11:34:23.921515 24710 tablet_bootstrap.cc:654] T c1152ff5766f4e2e90b97218a5723d20 P f87e487a91b542408b4a24ffc52421be: Neither blocks nor log segments found. Creating new log.
I20260810 11:34:23.921823 24712 tablet_bootstrap.cc:492] T c1152ff5766f4e2e90b97218a5723d20 P 8a82dd1f6f654360b68d7ff014fe5e71: Bootstrap starting.
I20260810 11:34:23.923128 24710 log.cc:824] T c1152ff5766f4e2e90b97218a5723d20 P f87e487a91b542408b4a24ffc52421be: Log is configured to *not* fsync() on all Append() calls
I20260810 11:34:23.924876 24711 tablet_bootstrap.cc:654] T c1152ff5766f4e2e90b97218a5723d20 P 1103ed0eb8974fd8bc3eb2955eacbc78: Neither blocks nor log segments found. Creating new log.
I20260810 11:34:23.925824 24712 tablet_bootstrap.cc:654] T c1152ff5766f4e2e90b97218a5723d20 P 8a82dd1f6f654360b68d7ff014fe5e71: Neither blocks nor log segments found. Creating new log.
I20260810 11:34:23.926509 24711 log.cc:824] T c1152ff5766f4e2e90b97218a5723d20 P 1103ed0eb8974fd8bc3eb2955eacbc78: Log is configured to *not* fsync() on all Append() calls
I20260810 11:34:23.927384 24710 tablet_bootstrap.cc:492] T c1152ff5766f4e2e90b97218a5723d20 P f87e487a91b542408b4a24ffc52421be: No bootstrap required, opened a new log
I20260810 11:34:23.927728 24710 ts_tablet_manager.cc:1397] T c1152ff5766f4e2e90b97218a5723d20 P f87e487a91b542408b4a24ffc52421be: Time spent bootstrapping tablet: real 0.012s user 0.008s sys 0.000s
I20260810 11:34:23.927845 24712 log.cc:824] T c1152ff5766f4e2e90b97218a5723d20 P 8a82dd1f6f654360b68d7ff014fe5e71: Log is configured to *not* fsync() on all Append() calls
I20260810 11:34:23.930784 24711 tablet_bootstrap.cc:492] T c1152ff5766f4e2e90b97218a5723d20 P 1103ed0eb8974fd8bc3eb2955eacbc78: No bootstrap required, opened a new log
I20260810 11:34:23.931121 24711 ts_tablet_manager.cc:1397] T c1152ff5766f4e2e90b97218a5723d20 P 1103ed0eb8974fd8bc3eb2955eacbc78: Time spent bootstrapping tablet: real 0.012s user 0.000s sys 0.008s
I20260810 11:34:23.931880 24712 tablet_bootstrap.cc:492] T c1152ff5766f4e2e90b97218a5723d20 P 8a82dd1f6f654360b68d7ff014fe5e71: No bootstrap required, opened a new log
I20260810 11:34:23.932202 24712 ts_tablet_manager.cc:1397] T c1152ff5766f4e2e90b97218a5723d20 P 8a82dd1f6f654360b68d7ff014fe5e71: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.003s
I20260810 11:34:23.939857 24710 raft_consensus.cc:348] T c1152ff5766f4e2e90b97218a5723d20 P f87e487a91b542408b4a24ffc52421be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f87e487a91b542408b4a24ffc52421be" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 40573 } } peers { permanent_uuid: "8a82dd1f6f654360b68d7ff014fe5e71" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 43511 } } peers { permanent_uuid: "1103ed0eb8974fd8bc3eb2955eacbc78" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 34511 } }
I20260810 11:34:23.940447 24710 raft_consensus.cc:374] T c1152ff5766f4e2e90b97218a5723d20 P f87e487a91b542408b4a24ffc52421be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:34:23.940565 24710 raft_consensus.cc:729] T c1152ff5766f4e2e90b97218a5723d20 P f87e487a91b542408b4a24ffc52421be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f87e487a91b542408b4a24ffc52421be, State: Initialized, Role: FOLLOWER
I20260810 11:34:23.940761 24712 raft_consensus.cc:348] T c1152ff5766f4e2e90b97218a5723d20 P 8a82dd1f6f654360b68d7ff014fe5e71 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f87e487a91b542408b4a24ffc52421be" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 40573 } } peers { permanent_uuid: "8a82dd1f6f654360b68d7ff014fe5e71" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 43511 } } peers { permanent_uuid: "1103ed0eb8974fd8bc3eb2955eacbc78" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 34511 } }
I20260810 11:34:23.941308 24712 raft_consensus.cc:374] T c1152ff5766f4e2e90b97218a5723d20 P 8a82dd1f6f654360b68d7ff014fe5e71 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:34:23.941506 24712 raft_consensus.cc:729] T c1152ff5766f4e2e90b97218a5723d20 P 8a82dd1f6f654360b68d7ff014fe5e71 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a82dd1f6f654360b68d7ff014fe5e71, State: Initialized, Role: FOLLOWER
I20260810 11:34:23.941309 24710 consensus_queue.cc:260] T c1152ff5766f4e2e90b97218a5723d20 P f87e487a91b542408b4a24ffc52421be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f87e487a91b542408b4a24ffc52421be" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 40573 } } peers { permanent_uuid: "8a82dd1f6f654360b68d7ff014fe5e71" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 43511 } } peers { permanent_uuid: "1103ed0eb8974fd8bc3eb2955eacbc78" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 34511 } }
I20260810 11:34:23.942198 24712 consensus_queue.cc:260] T c1152ff5766f4e2e90b97218a5723d20 P 8a82dd1f6f654360b68d7ff014fe5e71 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f87e487a91b542408b4a24ffc52421be" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 40573 } } peers { permanent_uuid: "8a82dd1f6f654360b68d7ff014fe5e71" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 43511 } } peers { permanent_uuid: "1103ed0eb8974fd8bc3eb2955eacbc78" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 34511 } }
I20260810 11:34:23.943010 24711 raft_consensus.cc:348] T c1152ff5766f4e2e90b97218a5723d20 P 1103ed0eb8974fd8bc3eb2955eacbc78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f87e487a91b542408b4a24ffc52421be" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 40573 } } peers { permanent_uuid: "8a82dd1f6f654360b68d7ff014fe5e71" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 43511 } } peers { permanent_uuid: "1103ed0eb8974fd8bc3eb2955eacbc78" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 34511 } }
I20260810 11:34:23.943624 24711 raft_consensus.cc:374] T c1152ff5766f4e2e90b97218a5723d20 P 1103ed0eb8974fd8bc3eb2955eacbc78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:34:23.943745 24711 raft_consensus.cc:729] T c1152ff5766f4e2e90b97218a5723d20 P 1103ed0eb8974fd8bc3eb2955eacbc78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1103ed0eb8974fd8bc3eb2955eacbc78, State: Initialized, Role: FOLLOWER
I20260810 11:34:23.944339 24712 ts_tablet_manager.cc:1428] T c1152ff5766f4e2e90b97218a5723d20 P 8a82dd1f6f654360b68d7ff014fe5e71: Time spent starting tablet: real 0.012s user 0.004s sys 0.008s
I20260810 11:34:23.944442 24711 consensus_queue.cc:260] T c1152ff5766f4e2e90b97218a5723d20 P 1103ed0eb8974fd8bc3eb2955eacbc78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f87e487a91b542408b4a24ffc52421be" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 40573 } } peers { permanent_uuid: "8a82dd1f6f654360b68d7ff014fe5e71" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 43511 } } peers { permanent_uuid: "1103ed0eb8974fd8bc3eb2955eacbc78" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 34511 } }
I20260810 11:34:23.957710 24554 heartbeater.cc:503] Master 127.8.18.254:42587 was elected leader, sending a full tablet report...
I20260810 11:34:23.958709 24711 ts_tablet_manager.cc:1428] T c1152ff5766f4e2e90b97218a5723d20 P 1103ed0eb8974fd8bc3eb2955eacbc78: Time spent starting tablet: real 0.027s user 0.013s sys 0.016s
I20260810 11:34:23.960482 24690 heartbeater.cc:503] Master 127.8.18.254:42587 was elected leader, sending a full tablet report...
I20260810 11:34:23.961059 24710 ts_tablet_manager.cc:1428] T c1152ff5766f4e2e90b97218a5723d20 P f87e487a91b542408b4a24ffc52421be: Time spent starting tablet: real 0.033s user 0.020s sys 0.000s
I20260810 11:34:23.965755 24718 raft_consensus.cc:482] T c1152ff5766f4e2e90b97218a5723d20 P 1103ed0eb8974fd8bc3eb2955eacbc78 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 11:34:23.966120 24718 raft_consensus.cc:504] T c1152ff5766f4e2e90b97218a5723d20 P 1103ed0eb8974fd8bc3eb2955eacbc78 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f87e487a91b542408b4a24ffc52421be" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 40573 } } peers { permanent_uuid: "8a82dd1f6f654360b68d7ff014fe5e71" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 43511 } } peers { permanent_uuid: "1103ed0eb8974fd8bc3eb2955eacbc78" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 34511 } }
I20260810 11:34:23.968330 24718 leader_election.cc:288] T c1152ff5766f4e2e90b97218a5723d20 P 1103ed0eb8974fd8bc3eb2955eacbc78 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f87e487a91b542408b4a24ffc52421be (127.8.18.195:40573), 8a82dd1f6f654360b68d7ff014fe5e71 (127.8.18.193:43511)
I20260810 11:34:23.976255 24707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:34:23.968634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.194:58145 (local address 127.8.18.195:40573)
0810 11:34:23.968907 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0810 11:34:23.968916 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 11:34:23.970566 (+ 1650us) server_negotiation.cc:407] Connection header received
0810 11:34:23.970784 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:34:23.970789 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:34:23.970865 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:34:23.970977 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:34:23.971463 (+ 486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:34:23.972428 (+ 965us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:34:23.974334 (+ 1906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:34:23.975150 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:34:23.975364 (+ 214us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:34:23.975458 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0810 11:34:23.975832 (+ 374us) server_negotiation.cc:299] Negotiation successful
0810 11:34:23.975978 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260810 11:34:23.976334 24706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:34:23.969348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.194:42041 (local address 127.8.18.193:43511)
0810 11:34:23.969586 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0810 11:34:23.969590 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 11:34:23.969691 (+ 101us) server_negotiation.cc:407] Connection header received
0810 11:34:23.969959 (+ 268us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:34:23.969962 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:34:23.970025 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:34:23.970204 (+ 179us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:34:23.970964 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:34:23.972086 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:34:23.974227 (+ 2141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:34:23.975184 (+ 957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:34:23.975408 (+ 224us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:34:23.975491 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0810 11:34:23.975911 (+ 420us) server_negotiation.cc:299] Negotiation successful
0810 11:34:23.976082 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260810 11:34:23.976423 24720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:34:23.968745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.195:40573 (local address 127.8.18.194:58145)
0810 11:34:23.970433 (+ 1688us) negotiation.cc:107] Waiting for socket to connect
0810 11:34:23.970459 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0810 11:34:23.970619 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:34:23.970995 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:34:23.970998 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:34:23.971019 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:34:23.971311 (+ 292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:34:23.971317 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:34:23.972568 (+ 1251us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:34:23.972573 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:34:23.974154 (+ 1581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:34:23.974169 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:34:23.975420 (+ 1251us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:34:23.975428 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:34:23.975626 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:34:23.975684 (+ 58us) client_negotiation.cc:769] Sending connection context
0810 11:34:23.975801 (+ 117us) client_negotiation.cc:240] Negotiation successful
0810 11:34:23.975956 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1564,"mutex_wait_us":41,"thread_start_us":1359,"threads_started":1}
I20260810 11:34:23.977170 24719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:34:23.968669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.193:43511 (local address 127.8.18.194:42041)
0810 11:34:23.969570 (+ 901us) negotiation.cc:107] Waiting for socket to connect
0810 11:34:23.969598 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0810 11:34:23.969808 (+ 210us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:34:23.970274 (+ 466us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:34:23.970283 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:34:23.970328 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:34:23.970781 (+ 453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:34:23.970795 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:34:23.972226 (+ 1431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:34:23.972230 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:34:23.973984 (+ 1754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:34:23.974000 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:34:23.975453 (+ 1453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:34:23.975460 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:34:23.975693 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:34:23.975734 (+ 41us) client_negotiation.cc:769] Sending connection context
0810 11:34:23.976627 (+ 893us) client_negotiation.cc:240] Negotiation successful
0810 11:34:23.976807 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":655,"mutex_wait_us":36,"thread_start_us":411,"threads_started":1}
I20260810 11:34:23.977766 24645 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1152ff5766f4e2e90b97218a5723d20" candidate_uuid: "1103ed0eb8974fd8bc3eb2955eacbc78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f87e487a91b542408b4a24ffc52421be" is_pre_election: true
I20260810 11:34:23.978255 24645 raft_consensus.cc:2454] T c1152ff5766f4e2e90b97218a5723d20 P f87e487a91b542408b4a24ffc52421be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1103ed0eb8974fd8bc3eb2955eacbc78 in term 0.
I20260810 11:34:23.978931 24442 leader_election.cc:302] T c1152ff5766f4e2e90b97218a5723d20 P 1103ed0eb8974fd8bc3eb2955eacbc78 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1103ed0eb8974fd8bc3eb2955eacbc78, f87e487a91b542408b4a24ffc52421be; no voters:
I20260810 11:34:23.979389 24718 raft_consensus.cc:2781] T c1152ff5766f4e2e90b97218a5723d20 P 1103ed0eb8974fd8bc3eb2955eacbc78 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 11:34:23.979560 24718 raft_consensus.cc:482] T c1152ff5766f4e2e90b97218a5723d20 P 1103ed0eb8974fd8bc3eb2955eacbc78 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 11:34:23.979683 24718 raft_consensus.cc:3037] T c1152ff5766f4e2e90b97218a5723d20 P 1103ed0eb8974fd8bc3eb2955eacbc78 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:34:23.979619 24370 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1152ff5766f4e2e90b97218a5723d20" candidate_uuid: "1103ed0eb8974fd8bc3eb2955eacbc78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a82dd1f6f654360b68d7ff014fe5e71" is_pre_election: true
I20260810 11:34:23.980214 24370 raft_consensus.cc:2454] T c1152ff5766f4e2e90b97218a5723d20 P 8a82dd1f6f654360b68d7ff014fe5e71 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1103ed0eb8974fd8bc3eb2955eacbc78 in term 0.
I20260810 11:34:23.982400 24718 raft_consensus.cc:504] T c1152ff5766f4e2e90b97218a5723d20 P 1103ed0eb8974fd8bc3eb2955eacbc78 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f87e487a91b542408b4a24ffc52421be" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 40573 } } peers { permanent_uuid: "8a82dd1f6f654360b68d7ff014fe5e71" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 43511 } } peers { permanent_uuid: "1103ed0eb8974fd8bc3eb2955eacbc78" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 34511 } }
I20260810 11:34:23.983127 24718 leader_election.cc:288] T c1152ff5766f4e2e90b97218a5723d20 P 1103ed0eb8974fd8bc3eb2955eacbc78 [CANDIDATE]: Term 1 election: Requested vote from peers f87e487a91b542408b4a24ffc52421be (127.8.18.195:40573), 8a82dd1f6f654360b68d7ff014fe5e71 (127.8.18.193:43511)
I20260810 11:34:23.983701 24370 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1152ff5766f4e2e90b97218a5723d20" candidate_uuid: "1103ed0eb8974fd8bc3eb2955eacbc78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a82dd1f6f654360b68d7ff014fe5e71"
I20260810 11:34:23.983721 24645 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1152ff5766f4e2e90b97218a5723d20" candidate_uuid: "1103ed0eb8974fd8bc3eb2955eacbc78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f87e487a91b542408b4a24ffc52421be"
I20260810 11:34:23.983942 24370 raft_consensus.cc:3037] T c1152ff5766f4e2e90b97218a5723d20 P 8a82dd1f6f654360b68d7ff014fe5e71 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:34:23.983968 24645 raft_consensus.cc:3037] T c1152ff5766f4e2e90b97218a5723d20 P f87e487a91b542408b4a24ffc52421be [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:34:23.986614 24370 raft_consensus.cc:2454] T c1152ff5766f4e2e90b97218a5723d20 P 8a82dd1f6f654360b68d7ff014fe5e71 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1103ed0eb8974fd8bc3eb2955eacbc78 in term 1.
I20260810 11:34:23.986814 24645 raft_consensus.cc:2454] T c1152ff5766f4e2e90b97218a5723d20 P f87e487a91b542408b4a24ffc52421be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1103ed0eb8974fd8bc3eb2955eacbc78 in term 1.
I20260810 11:34:23.987287 24441 leader_election.cc:302] T c1152ff5766f4e2e90b97218a5723d20 P 1103ed0eb8974fd8bc3eb2955eacbc78 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1103ed0eb8974fd8bc3eb2955eacbc78, 8a82dd1f6f654360b68d7ff014fe5e71; no voters:
I20260810 11:34:23.987691 24718 raft_consensus.cc:2781] T c1152ff5766f4e2e90b97218a5723d20 P 1103ed0eb8974fd8bc3eb2955eacbc78 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:34:23.988276 24718 raft_consensus.cc:686] T c1152ff5766f4e2e90b97218a5723d20 P 1103ed0eb8974fd8bc3eb2955eacbc78 [term 1 LEADER]: Becoming Leader. State: Replica: 1103ed0eb8974fd8bc3eb2955eacbc78, State: Running, Role: LEADER
I20260810 11:34:23.988860 24718 consensus_queue.cc:237] T c1152ff5766f4e2e90b97218a5723d20 P 1103ed0eb8974fd8bc3eb2955eacbc78 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f87e487a91b542408b4a24ffc52421be" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 40573 } } peers { permanent_uuid: "8a82dd1f6f654360b68d7ff014fe5e71" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 43511 } } peers { permanent_uuid: "1103ed0eb8974fd8bc3eb2955eacbc78" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 34511 } }
I20260810 11:34:23.994230 24222 catalog_manager.cc:5495] T c1152ff5766f4e2e90b97218a5723d20 P 1103ed0eb8974fd8bc3eb2955eacbc78 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1103ed0eb8974fd8bc3eb2955eacbc78 (127.8.18.194). New cstate: current_term: 1 leader_uuid: "1103ed0eb8974fd8bc3eb2955eacbc78" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f87e487a91b542408b4a24ffc52421be" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 40573 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a82dd1f6f654360b68d7ff014fe5e71" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 43511 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1103ed0eb8974fd8bc3eb2955eacbc78" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 34511 } health_report { overall_health: HEALTHY } } }
W20260810 11:34:24.034390 24417 tablet.cc:2369] T c1152ff5766f4e2e90b97218a5723d20 P 8a82dd1f6f654360b68d7ff014fe5e71: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 11:34:24.035553 24708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:34:24.029832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52990 (local address 127.8.18.194:34511)
0810 11:34:24.030059 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0810 11:34:24.030068 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 11:34:24.030090 (+ 22us) server_negotiation.cc:407] Connection header received
0810 11:34:24.030200 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:34:24.030203 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:34:24.030300 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:34:24.030469 (+ 169us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 11:34:24.030870 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:34:24.031624 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:34:24.033093 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:34:24.033588 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:34:24.033737 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:34:24.034283 (+ 546us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 11:34:24.034890 (+ 607us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 11:34:24.035054 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0810 11:34:24.035167 (+ 113us) server_negotiation.cc:299] Negotiation successful
0810 11:34:24.035260 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
W20260810 11:34:24.038897 24691 tablet.cc:2369] T c1152ff5766f4e2e90b97218a5723d20 P f87e487a91b542408b4a24ffc52421be: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 11:34:24.044919 24645 raft_consensus.cc:1261] T c1152ff5766f4e2e90b97218a5723d20 P f87e487a91b542408b4a24ffc52421be [term 1 FOLLOWER]: Refusing update from remote peer 1103ed0eb8974fd8bc3eb2955eacbc78: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 11:34:24.044920 24370 raft_consensus.cc:1261] T c1152ff5766f4e2e90b97218a5723d20 P 8a82dd1f6f654360b68d7ff014fe5e71 [term 1 FOLLOWER]: Refusing update from remote peer 1103ed0eb8974fd8bc3eb2955eacbc78: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 11:34:24.046082 24721 consensus_queue.cc:1035] T c1152ff5766f4e2e90b97218a5723d20 P 1103ed0eb8974fd8bc3eb2955eacbc78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f87e487a91b542408b4a24ffc52421be" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 40573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 11:34:24.046522 24718 consensus_queue.cc:1035] T c1152ff5766f4e2e90b97218a5723d20 P 1103ed0eb8974fd8bc3eb2955eacbc78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a82dd1f6f654360b68d7ff014fe5e71" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 43511 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 11:34:24.060557 24724 mvcc.cc:205] Tried to move back new op lower bound from 7316937375917166592 to 7316937375703851008. Current Snapshot: MvccSnapshot[applied={T|T < 7316937375917166592}]
I20260810 11:34:24.063903 24726 mvcc.cc:205] Tried to move back new op lower bound from 7316937375917166592 to 7316937375703851008. Current Snapshot: MvccSnapshot[applied={T|T < 7316937375917166592}]
I20260810 11:34:24.078609 24222 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55438:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 11:34:24.078972 24222 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55438:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 11:34:24.083197 24222 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 25ef7a60224c4f7bb5147d7c5ed8251c: Sending DeleteTablet for 3 replicas of tablet c1152ff5766f4e2e90b97218a5723d20
I20260810 11:34:24.084687 24488 tablet_service.cc:1514] Processing DeleteTablet for tablet c1152ff5766f4e2e90b97218a5723d20 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:34:24 UTC) from {username='kudu', principal='kudu/127.8.18.254@KRBTEST.COM'} at 127.0.0.1:52980
I20260810 11:34:24.084656 24350 tablet_service.cc:1514] Processing DeleteTablet for tablet c1152ff5766f4e2e90b97218a5723d20 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:34:24 UTC) from {username='kudu', principal='kudu/127.8.18.254@KRBTEST.COM'} at 127.0.0.1:59218
I20260810 11:34:24.084726 24625 tablet_service.cc:1514] Processing DeleteTablet for tablet c1152ff5766f4e2e90b97218a5723d20 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:34:24 UTC) from {username='kudu', principal='kudu/127.8.18.254@KRBTEST.COM'} at 127.0.0.1:58648
I20260810 11:34:24.086652 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 24285
I20260810 11:34:24.097811 24736 tablet_replica.cc:330] T c1152ff5766f4e2e90b97218a5723d20 P f87e487a91b542408b4a24ffc52421be: stopping tablet replica
I20260810 11:34:24.098498 24736 raft_consensus.cc:2229] T c1152ff5766f4e2e90b97218a5723d20 P f87e487a91b542408b4a24ffc52421be [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 11:34:24.099221 24736 raft_consensus.cc:2258] T c1152ff5766f4e2e90b97218a5723d20 P f87e487a91b542408b4a24ffc52421be [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:34:24.102337 24736 ts_tablet_manager.cc:1905] T c1152ff5766f4e2e90b97218a5723d20 P f87e487a91b542408b4a24ffc52421be: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:34:24.112305 24734 tablet_replica.cc:330] T c1152ff5766f4e2e90b97218a5723d20 P 1103ed0eb8974fd8bc3eb2955eacbc78: stopping tablet replica
I20260810 11:34:24.112299 24736 ts_tablet_manager.cc:1918] T c1152ff5766f4e2e90b97218a5723d20 P f87e487a91b542408b4a24ffc52421be: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 11:34:24.112520 24736 log.cc:1196] T c1152ff5766f4e2e90b97218a5723d20 P f87e487a91b542408b4a24ffc52421be: Deleting WAL directory at /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/ts-2/wal/wals/c1152ff5766f4e2e90b97218a5723d20
I20260810 11:34:24.112917 24734 raft_consensus.cc:2229] T c1152ff5766f4e2e90b97218a5723d20 P 1103ed0eb8974fd8bc3eb2955eacbc78 [term 1 LEADER]: Raft consensus shutting down.
I20260810 11:34:24.113128 24736 ts_tablet_manager.cc:1939] T c1152ff5766f4e2e90b97218a5723d20 P f87e487a91b542408b4a24ffc52421be: Deleting consensus metadata
I20260810 11:34:24.113772 24734 raft_consensus.cc:2258] T c1152ff5766f4e2e90b97218a5723d20 P 1103ed0eb8974fd8bc3eb2955eacbc78 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:34:24.115339 24209 catalog_manager.cc:4841] TS f87e487a91b542408b4a24ffc52421be (127.8.18.195:40573): tablet c1152ff5766f4e2e90b97218a5723d20 (table test-table [id=59d3a3102c4c4d56a327d4e9e4efc0ba]) successfully deleted
I20260810 11:34:24.116135 24735 tablet_replica.cc:330] T c1152ff5766f4e2e90b97218a5723d20 P 8a82dd1f6f654360b68d7ff014fe5e71: stopping tablet replica
I20260810 11:34:24.116756 24735 raft_consensus.cc:2229] T c1152ff5766f4e2e90b97218a5723d20 P 8a82dd1f6f654360b68d7ff014fe5e71 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 11:34:24.117440 24735 raft_consensus.cc:2258] T c1152ff5766f4e2e90b97218a5723d20 P 8a82dd1f6f654360b68d7ff014fe5e71 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:34:24.119305 24734 ts_tablet_manager.cc:1905] T c1152ff5766f4e2e90b97218a5723d20 P 1103ed0eb8974fd8bc3eb2955eacbc78: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:34:24.120141 24735 ts_tablet_manager.cc:1905] T c1152ff5766f4e2e90b97218a5723d20 P 8a82dd1f6f654360b68d7ff014fe5e71: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:34:24.128855 24735 ts_tablet_manager.cc:1918] T c1152ff5766f4e2e90b97218a5723d20 P 8a82dd1f6f654360b68d7ff014fe5e71: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 11:34:24.129092 24735 log.cc:1196] T c1152ff5766f4e2e90b97218a5723d20 P 8a82dd1f6f654360b68d7ff014fe5e71: Deleting WAL directory at /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/ts-0/wal/wals/c1152ff5766f4e2e90b97218a5723d20
I20260810 11:34:24.129820 24735 ts_tablet_manager.cc:1939] T c1152ff5766f4e2e90b97218a5723d20 P 8a82dd1f6f654360b68d7ff014fe5e71: Deleting consensus metadata
Aug 10 11:34:24 dist-test-slave-srgv krb5kdc[24177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361661, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.18.193@KRBTEST.COM
I20260810 11:34:24.130438 24734 ts_tablet_manager.cc:1918] T c1152ff5766f4e2e90b97218a5723d20 P 1103ed0eb8974fd8bc3eb2955eacbc78: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 11:34:24.130686 24734 log.cc:1196] T c1152ff5766f4e2e90b97218a5723d20 P 1103ed0eb8974fd8bc3eb2955eacbc78: Deleting WAL directory at /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/ts-1/wal/wals/c1152ff5766f4e2e90b97218a5723d20
I20260810 11:34:24.131409 24734 ts_tablet_manager.cc:1939] T c1152ff5766f4e2e90b97218a5723d20 P 1103ed0eb8974fd8bc3eb2955eacbc78: Deleting consensus metadata
I20260810 11:34:24.132964 24208 catalog_manager.cc:4841] TS 8a82dd1f6f654360b68d7ff014fe5e71 (127.8.18.193:43511): tablet c1152ff5766f4e2e90b97218a5723d20 (table test-table [id=59d3a3102c4c4d56a327d4e9e4efc0ba]) successfully deleted
I20260810 11:34:24.133562 24207 catalog_manager.cc:4841] TS 1103ed0eb8974fd8bc3eb2955eacbc78 (127.8.18.194:34511): tablet c1152ff5766f4e2e90b97218a5723d20 (table test-table [id=59d3a3102c4c4d56a327d4e9e4efc0ba]) successfully deleted
I20260810 11:34:24.140357 24706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:34:24.088364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59228 (local address 127.8.18.193:43511)
0810 11:34:24.088630 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0810 11:34:24.088639 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 11:34:24.108383 (+ 19744us) server_negotiation.cc:407] Connection header received
0810 11:34:24.109046 (+ 663us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:34:24.109055 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:34:24.109186 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:34:24.109278 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:34:24.114310 (+ 5032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:34:24.115558 (+ 1248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:34:24.117287 (+ 1729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:34:24.118077 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:34:24.118250 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:34:24.131958 (+ 13708us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:34:24.131979 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:34:24.132001 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:34:24.132066 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:34:24.136434 (+ 4368us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:34:24.137515 (+ 1081us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:34:24.137532 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:34:24.137537 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:34:24.137751 (+ 214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:34:24.138230 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:34:24.138237 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:34:24.138240 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:34:24.138772 (+ 532us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:34:24.139107 (+ 335us) server_negotiation.cc:1035] Waiting for connection context
0810 11:34:24.139600 (+ 493us) server_negotiation.cc:299] Negotiation successful
0810 11:34:24.139885 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260810 11:34:24.141355 24411 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:34:24.270864 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 24285
I20260810 11:34:24.294780 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 24423
Aug 10 11:34:24 dist-test-slave-srgv krb5kdc[24177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361661, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.18.194@KRBTEST.COM
I20260810 11:34:24.315872 24708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:34:24.295848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53006 (local address 127.8.18.194:34511)
0810 11:34:24.296078 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0810 11:34:24.296086 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:34:24.296110 (+ 24us) server_negotiation.cc:407] Connection header received
0810 11:34:24.296183 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:34:24.296189 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:34:24.296307 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:34:24.296428 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:34:24.298012 (+ 1584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:34:24.299387 (+ 1375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:34:24.300934 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:34:24.301745 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:34:24.301904 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:34:24.307399 (+ 5495us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:34:24.307424 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:34:24.307447 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:34:24.307519 (+ 72us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:34:24.312102 (+ 4583us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:34:24.313112 (+ 1010us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:34:24.313124 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:34:24.313129 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:34:24.313351 (+ 222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:34:24.313956 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:34:24.313962 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:34:24.313966 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:34:24.314475 (+ 509us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:34:24.314838 (+ 363us) server_negotiation.cc:1035] Waiting for connection context
0810 11:34:24.315162 (+ 324us) server_negotiation.cc:299] Negotiation successful
0810 11:34:24.315445 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260810 11:34:24.316843 24549 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:34:24.439329 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 24423
I20260810 11:34:24.462328 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 24560
Aug 10 11:34:24 dist-test-slave-srgv krb5kdc[24177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361661, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.18.195@KRBTEST.COM
I20260810 11:34:24.480185 24707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:34:24.463290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58654 (local address 127.8.18.195:40573)
0810 11:34:24.463506 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0810 11:34:24.463510 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 11:34:24.463531 (+ 21us) server_negotiation.cc:407] Connection header received
0810 11:34:24.463629 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:34:24.463633 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:34:24.463700 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:34:24.463806 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:34:24.465272 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:34:24.466173 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:34:24.467826 (+ 1653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:34:24.468284 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:34:24.468397 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:34:24.473645 (+ 5248us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:34:24.473665 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:34:24.473683 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:34:24.473727 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:34:24.476836 (+ 3109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:34:24.477886 (+ 1050us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:34:24.477895 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:34:24.477897 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:34:24.478057 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:34:24.478534 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:34:24.478538 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:34:24.478540 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:34:24.478992 (+ 452us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:34:24.479294 (+ 302us) server_negotiation.cc:1035] Waiting for connection context
0810 11:34:24.479655 (+ 361us) server_negotiation.cc:299] Negotiation successful
0810 11:34:24.479859 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260810 11:34:24.481045 24685 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:34:24.577920 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 24560
I20260810 11:34:24.598517 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 24190
I20260810 11:34:24.599871 24253 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:34:24.722048 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 24190
I20260810 11:34:44.748521 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.18.254:42587
--webserver_interface=127.8.18.254
--webserver_port=39739
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.254.keytab
--principal=kudu/127.8.18.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.18.212:33987
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.18.254:42587
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:34:45.138236 24742 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:34:45.138592 24742 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:34:45.138689 24742 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:34:45.146718 24742 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260810 11:34:45.146857 24742 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260810 11:34:45.146898 24742 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260810 11:34:45.146947 24742 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 11:34:45.147004 24742 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:34:45.147084 24742 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:34:45.147142 24742 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 11:34:45.147192 24742 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 11:34:45.157963 24742 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.18.212:33987
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.8.18.254:42587
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.254.keytab
--principal=kudu/127.8.18.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.18.254:42587
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.18.254
--webserver_port=39739
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:34:45.159339 24742 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:34:45.160735 24742 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:34:45.188407 24751 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:34:45.188427 24748 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:34:45.188489 24749 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:34:45.188858 24742 server_base.cc:1034] running on GCE node
I20260810 11:34:45.190197 24742 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:34:45.191969 24742 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:34:45.193229 24742 hybrid_clock.cc:648] HybridClock initialized: now 1786361685193163 us; error 37 us; skew 500 ppm
I20260810 11:34:45.193729 24742 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:34:45 dist-test-slave-srgv krb5kdc[24177](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361685, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:34:45.197937 24742 init.cc:377] Logged in from keytab as kudu/127.8.18.254@KRBTEST.COM (short username kudu)
I20260810 11:34:45.199965 24742 webserver.cc:460] Webserver started at http://127.8.18.254:39739/ using document root <none> and password file <none>
I20260810 11:34:45.200834 24742 fs_manager.cc:359] Metadata directory not provided
I20260810 11:34:45.200955 24742 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:34:45.208757 24742 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.005s sys 0.000s
I20260810 11:34:45.213392 24758 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:34:45.215160 24742 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260810 11:34:45.215421 24742 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/master-0/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/master-0/wal
uuid: "25ef7a60224c4f7bb5147d7c5ed8251c"
format_stamp: "Formatted at 2026-08-10 11:34:21 on dist-test-slave-srgv"
server_key: "59e635f02b278030db5011de502297ee"
server_key_iv: "b7d235b3fe008091868335d912f66813"
server_key_version: "encryptionkey@0"
I20260810 11:34:45.216738 24742 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:34:45.240655 24742 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:34:45.246847 24742 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:34:45.247248 24742 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:34:45.271404 24742 rpc_server.cc:307] RPC server started. Bound to: 127.8.18.254:42587
I20260810 11:34:45.271430 24810 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.18.254:42587 every 8 connection(s)
I20260810 11:34:45.273577 24742 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
I20260810 11:34:45.282272 24811 sys_catalog.cc:263] Verifying existing consensus state
I20260810 11:34:45.282934 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 24742
I20260810 11:34:45.283849 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.18.193:43511
--local_ip_for_outbound_sockets=127.8.18.193
--tserver_master_addrs=127.8.18.254:42587
--webserver_port=38413
--webserver_interface=127.8.18.193
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.193.keytab
--principal=kudu/127.8.18.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.18.212:33987
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 11:34:45.286535 24811 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 25ef7a60224c4f7bb5147d7c5ed8251c: Bootstrap starting.
I20260810 11:34:45.310777 24811 log.cc:824] T 00000000000000000000000000000000 P 25ef7a60224c4f7bb5147d7c5ed8251c: Log is configured to *not* fsync() on all Append() calls
I20260810 11:34:45.325096 24811 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 25ef7a60224c4f7bb5147d7c5ed8251c: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 11:34:45.325939 24811 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 25ef7a60224c4f7bb5147d7c5ed8251c: Bootstrap complete.
I20260810 11:34:45.336376 24811 raft_consensus.cc:348] T 00000000000000000000000000000000 P 25ef7a60224c4f7bb5147d7c5ed8251c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25ef7a60224c4f7bb5147d7c5ed8251c" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 42587 } }
I20260810 11:34:45.337446 24811 raft_consensus.cc:729] T 00000000000000000000000000000000 P 25ef7a60224c4f7bb5147d7c5ed8251c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25ef7a60224c4f7bb5147d7c5ed8251c, State: Initialized, Role: FOLLOWER
I20260810 11:34:45.338266 24811 consensus_queue.cc:260] T 00000000000000000000000000000000 P 25ef7a60224c4f7bb5147d7c5ed8251c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25ef7a60224c4f7bb5147d7c5ed8251c" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 42587 } }
I20260810 11:34:45.338557 24811 raft_consensus.cc:388] T 00000000000000000000000000000000 P 25ef7a60224c4f7bb5147d7c5ed8251c [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 11:34:45.338714 24811 raft_consensus.cc:482] T 00000000000000000000000000000000 P 25ef7a60224c4f7bb5147d7c5ed8251c [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 11:34:45.338917 24811 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 25ef7a60224c4f7bb5147d7c5ed8251c [term 1 FOLLOWER]: Advancing to term 2
I20260810 11:34:45.343339 24811 raft_consensus.cc:504] T 00000000000000000000000000000000 P 25ef7a60224c4f7bb5147d7c5ed8251c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25ef7a60224c4f7bb5147d7c5ed8251c" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 42587 } }
I20260810 11:34:45.344064 24811 leader_election.cc:302] T 00000000000000000000000000000000 P 25ef7a60224c4f7bb5147d7c5ed8251c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 25ef7a60224c4f7bb5147d7c5ed8251c; no voters:
I20260810 11:34:45.344835 24811 leader_election.cc:288] T 00000000000000000000000000000000 P 25ef7a60224c4f7bb5147d7c5ed8251c [CANDIDATE]: Term 2 election: Requested vote from peers
I20260810 11:34:45.345021 24815 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 25ef7a60224c4f7bb5147d7c5ed8251c [term 2 FOLLOWER]: Leader election won for term 2
I20260810 11:34:45.346583 24815 raft_consensus.cc:686] T 00000000000000000000000000000000 P 25ef7a60224c4f7bb5147d7c5ed8251c [term 2 LEADER]: Becoming Leader. State: Replica: 25ef7a60224c4f7bb5147d7c5ed8251c, State: Running, Role: LEADER
I20260810 11:34:45.347254 24815 consensus_queue.cc:237] T 00000000000000000000000000000000 P 25ef7a60224c4f7bb5147d7c5ed8251c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25ef7a60224c4f7bb5147d7c5ed8251c" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 42587 } }
I20260810 11:34:45.347647 24811 sys_catalog.cc:564] T 00000000000000000000000000000000 P 25ef7a60224c4f7bb5147d7c5ed8251c [sys.catalog]: configured and running, proceeding with master startup.
I20260810 11:34:45.350337 24816 sys_catalog.cc:455] T 00000000000000000000000000000000 P 25ef7a60224c4f7bb5147d7c5ed8251c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "25ef7a60224c4f7bb5147d7c5ed8251c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25ef7a60224c4f7bb5147d7c5ed8251c" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 42587 } } }
I20260810 11:34:45.350831 24816 sys_catalog.cc:458] T 00000000000000000000000000000000 P 25ef7a60224c4f7bb5147d7c5ed8251c [sys.catalog]: This master's current role is: LEADER
I20260810 11:34:45.351385 24817 sys_catalog.cc:455] T 00000000000000000000000000000000 P 25ef7a60224c4f7bb5147d7c5ed8251c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 25ef7a60224c4f7bb5147d7c5ed8251c. Latest consensus state: current_term: 2 leader_uuid: "25ef7a60224c4f7bb5147d7c5ed8251c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25ef7a60224c4f7bb5147d7c5ed8251c" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 42587 } } }
I20260810 11:34:45.351722 24817 sys_catalog.cc:458] T 00000000000000000000000000000000 P 25ef7a60224c4f7bb5147d7c5ed8251c [sys.catalog]: This master's current role is: LEADER
I20260810 11:34:45.362762 24821 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 11:34:45.372439 24821 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 11:34:45.374207 24821 catalog_manager.cc:1259] Loaded cluster ID: e2a8e20598364125b81bb3c2699a96d8
I20260810 11:34:45.374434 24821 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 11:34:45.379034 24821 catalog_manager.cc:1504] Loading token signing keys...
I20260810 11:34:45.379316 24832 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260810 11:34:45.382162 24821 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 25ef7a60224c4f7bb5147d7c5ed8251c: Loaded TSK: 0
I20260810 11:34:45.416288 24821 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 25ef7a60224c4f7bb5147d7c5ed8251c: Generated new TSK 1
I20260810 11:34:45.416837 24821 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260810 11:34:45.695406 24813 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:34:45.695727 24813 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:34:45.695892 24813 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:34:45.704042 24813 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:34:45.704178 24813 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:34:45.704366 24813 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.18.193
I20260810 11:34:45.715723 24813 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.18.212:33987
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.193.keytab
--principal=kudu/127.8.18.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.18.193:43511
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.18.193
--webserver_port=38413
--tserver_master_addrs=127.8.18.254:42587
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.18.193
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:34:45.717275 24813 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:34:45.718894 24813 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:34:45.736383 24839 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:34:45.736344 24838 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:34:45.737771 24841 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:34:45.739753 24813 server_base.cc:1034] running on GCE node
I20260810 11:34:45.740569 24813 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:34:45.741855 24813 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:34:45.743162 24813 hybrid_clock.cc:648] HybridClock initialized: now 1786361685743098 us; error 65 us; skew 500 ppm
I20260810 11:34:45.743549 24813 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:34:45 dist-test-slave-srgv krb5kdc[24177](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361685, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:34:45.748109 24813 init.cc:377] Logged in from keytab as kudu/127.8.18.193@KRBTEST.COM (short username kudu)
I20260810 11:34:45.750296 24813 webserver.cc:460] Webserver started at http://127.8.18.193:38413/ using document root <none> and password file <none>
I20260810 11:34:45.751140 24813 fs_manager.cc:359] Metadata directory not provided
I20260810 11:34:45.751333 24813 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:34:45.757594 24813 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260810 11:34:45.760768 24848 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:34:45.762333 24813 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 11:34:45.762538 24813 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/ts-0/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/ts-0/wal
uuid: "8a82dd1f6f654360b68d7ff014fe5e71"
format_stamp: "Formatted at 2026-08-10 11:34:22 on dist-test-slave-srgv"
server_key: "d17b1c80112eb8b90b6e989b0c672614"
server_key_iv: "bb6e0401ef0735452bbcac4e2c4c779f"
server_key_version: "encryptionkey@0"
I20260810 11:34:45.763708 24813 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:34:45.780786 24813 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:34:45.786046 24813 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:34:45.786384 24813 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:34:45.787528 24813 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:34:45.789077 24813 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:34:45.789230 24813 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:34:45.789362 24813 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:34:45.789486 24813 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
I20260810 11:34:45.828233 24813 rpc_server.cc:307] RPC server started. Bound to: 127.8.18.193:43511
I20260810 11:34:45.828281 24961 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.18.193:43511 every 8 connection(s)
I20260810 11:34:45.829972 24813 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/ts-0/data/info.pb
I20260810 11:34:45.837109 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 24813
I20260810 11:34:45.837886 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.18.194:34511
--local_ip_for_outbound_sockets=127.8.18.194
--tserver_master_addrs=127.8.18.254:42587
--webserver_port=44741
--webserver_interface=127.8.18.194
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.194.keytab
--principal=kudu/127.8.18.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.18.212:33987
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 11:34:45 dist-test-slave-srgv krb5kdc[24177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361685, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.193@KRBTEST.COM for kudu/127.8.18.254@KRBTEST.COM
I20260810 11:34:45.858647 24964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:34:45.833184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.193:34511 (local address 127.8.18.254:42587)
0810 11:34:45.833935 (+ 751us) server_negotiation.cc:206] Beginning negotiation
0810 11:34:45.833958 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0810 11:34:45.834554 (+ 596us) server_negotiation.cc:407] Connection header received
0810 11:34:45.837229 (+ 2675us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:34:45.837243 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:34:45.837776 (+ 533us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:34:45.838207 (+ 431us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:34:45.840157 (+ 1950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:34:45.841631 (+ 1474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:34:45.843529 (+ 1898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:34:45.844290 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:34:45.844560 (+ 270us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:34:45.849012 (+ 4452us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:34:45.849028 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:34:45.849054 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:34:45.849117 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:34:45.852940 (+ 3823us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:34:45.854111 (+ 1171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:34:45.854121 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:34:45.854127 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:34:45.854344 (+ 217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:34:45.854893 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:34:45.854897 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:34:45.854900 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:34:45.855480 (+ 580us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:34:45.855704 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0810 11:34:45.856667 (+ 963us) server_negotiation.cc:299] Negotiation successful
0810 11:34:45.857193 (+ 526us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":468,"thread_start_us":307,"threads_started":1}
I20260810 11:34:45.858644 24965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:34:45.833421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:42587 (local address 127.8.18.193:34511)
0810 11:34:45.834346 (+ 925us) negotiation.cc:107] Waiting for socket to connect
0810 11:34:45.834395 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0810 11:34:45.836511 (+ 2116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:34:45.838537 (+ 2026us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:34:45.838549 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:34:45.839224 (+ 675us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:34:45.839853 (+ 629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:34:45.839876 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:34:45.841876 (+ 2000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:34:45.841881 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:34:45.843306 (+ 1425us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:34:45.843326 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:34:45.844622 (+ 1296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:34:45.844631 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:34:45.844957 (+ 326us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:34:45.845835 (+ 878us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:34:45.845861 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:34:45.848546 (+ 2685us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:34:45.853177 (+ 4631us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:34:45.853195 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:34:45.853209 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:34:45.853878 (+ 669us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:34:45.854512 (+ 634us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:34:45.854516 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:34:45.854518 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:34:45.854741 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:34:45.855658 (+ 917us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:34:45.855667 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:34:45.856050 (+ 383us) client_negotiation.cc:769] Sending connection context
0810 11:34:45.856354 (+ 304us) client_negotiation.cc:240] Negotiation successful
0810 11:34:45.857027 (+ 673us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":601,"thread_start_us":359,"threads_started":1}
I20260810 11:34:45.862331 24962 heartbeater.cc:347] Connected to a master server at 127.8.18.254:42587
I20260810 11:34:45.862787 24962 heartbeater.cc:464] Registering TS with master...
I20260810 11:34:45.863690 24962 heartbeater.cc:511] Master 127.8.18.254:42587 requested a full tablet report, sending...
I20260810 11:34:45.866652 24775 ts_manager.cc:194] Registered new tserver with Master: 8a82dd1f6f654360b68d7ff014fe5e71 (127.8.18.193:43511)
I20260810 11:34:45.869225 24775 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.18.193@KRBTEST.COM'} at 127.8.18.193:34511
W20260810 11:34:46.225170 24966 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:34:46.225605 24966 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:34:46.225775 24966 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:34:46.234020 24966 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:34:46.234195 24966 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:34:46.234376 24966 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.18.194
I20260810 11:34:46.245093 24966 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.18.212:33987
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.194.keytab
--principal=kudu/127.8.18.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.18.194:34511
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.18.194
--webserver_port=44741
--tserver_master_addrs=127.8.18.254:42587
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.18.194
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:34:46.246649 24966 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:34:46.248106 24966 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:34:46.262858 24974 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:34:46.262903 24972 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:34:46.263586 24971 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:34:46.264062 24966 server_base.cc:1034] running on GCE node
I20260810 11:34:46.264719 24966 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:34:46.266022 24966 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:34:46.267297 24966 hybrid_clock.cc:648] HybridClock initialized: now 1786361686267208 us; error 70 us; skew 500 ppm
I20260810 11:34:46.267724 24966 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:34:46 dist-test-slave-srgv krb5kdc[24177](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361686, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:34:46.271826 24966 init.cc:377] Logged in from keytab as kudu/127.8.18.194@KRBTEST.COM (short username kudu)
I20260810 11:34:46.273738 24966 webserver.cc:460] Webserver started at http://127.8.18.194:44741/ using document root <none> and password file <none>
I20260810 11:34:46.274523 24966 fs_manager.cc:359] Metadata directory not provided
I20260810 11:34:46.274712 24966 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:34:46.281113 24966 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.003s
I20260810 11:34:46.284739 24981 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:34:46.286808 24966 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260810 11:34:46.287048 24966 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/ts-1/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/ts-1/wal
uuid: "1103ed0eb8974fd8bc3eb2955eacbc78"
format_stamp: "Formatted at 2026-08-10 11:34:23 on dist-test-slave-srgv"
server_key: "bddc7322a41df9efaed932734d963059"
server_key_iv: "1fa90db762df1c07e2bda8f2688f461b"
server_key_version: "encryptionkey@0"
I20260810 11:34:46.288321 24966 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:34:46.310341 24966 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:34:46.315521 24966 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:34:46.315872 24966 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:34:46.316996 24966 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:34:46.318693 24966 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:34:46.318828 24966 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:34:46.318972 24966 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:34:46.319072 24966 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:34:46.357872 24966 rpc_server.cc:307] RPC server started. Bound to: 127.8.18.194:34511
I20260810 11:34:46.357936 25095 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.18.194:34511 every 8 connection(s)
I20260810 11:34:46.359760 24966 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/ts-1/data/info.pb
I20260810 11:34:46.363710 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 24966
I20260810 11:34:46.364634 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.18.195:40573
--local_ip_for_outbound_sockets=127.8.18.195
--tserver_master_addrs=127.8.18.254:42587
--webserver_port=46223
--webserver_interface=127.8.18.195
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.195.keytab
--principal=kudu/127.8.18.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.18.212:33987
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 11:34:46 dist-test-slave-srgv krb5kdc[24177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361686, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.194@KRBTEST.COM for kudu/127.8.18.254@KRBTEST.COM
I20260810 11:34:46.386904 25098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:34:46.362743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.194:54793 (local address 127.8.18.254:42587)
0810 11:34:46.363301 (+ 558us) server_negotiation.cc:206] Beginning negotiation
0810 11:34:46.363308 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:34:46.364536 (+ 1228us) server_negotiation.cc:407] Connection header received
0810 11:34:46.367761 (+ 3225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:34:46.367766 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:34:46.367863 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:34:46.367968 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:34:46.370490 (+ 2522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:34:46.371292 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:34:46.373927 (+ 2635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:34:46.374743 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:34:46.374919 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:34:46.379416 (+ 4497us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:34:46.379423 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:34:46.379426 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:34:46.379468 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:34:46.382584 (+ 3116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:34:46.383903 (+ 1319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:34:46.383909 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:34:46.383911 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:34:46.384079 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:34:46.384687 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:34:46.384692 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:34:46.384695 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:34:46.385009 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:34:46.385136 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0810 11:34:46.386300 (+ 1164us) server_negotiation.cc:299] Negotiation successful
0810 11:34:46.386532 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":398,"thread_start_us":250,"threads_started":1}
I20260810 11:34:46.388563 25099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:34:46.363322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:42587 (local address 127.8.18.194:54793)
0810 11:34:46.364322 (+ 1000us) negotiation.cc:107] Waiting for socket to connect
0810 11:34:46.364383 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0810 11:34:46.367347 (+ 2964us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:34:46.368351 (+ 1004us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:34:46.368365 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:34:46.369259 (+ 894us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:34:46.370207 (+ 948us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:34:46.370240 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:34:46.371503 (+ 1263us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:34:46.371511 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:34:46.373736 (+ 2225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:34:46.373752 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:34:46.375034 (+ 1282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:34:46.375040 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:34:46.375312 (+ 272us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:34:46.376107 (+ 795us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:34:46.376163 (+ 56us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:34:46.379036 (+ 2873us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:34:46.382829 (+ 3793us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:34:46.382844 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:34:46.382868 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:34:46.383710 (+ 842us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:34:46.384218 (+ 508us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:34:46.384226 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:34:46.384232 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:34:46.384534 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:34:46.385179 (+ 645us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:34:46.385192 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:34:46.385799 (+ 607us) client_negotiation.cc:769] Sending connection context
0810 11:34:46.386169 (+ 370us) client_negotiation.cc:240] Negotiation successful
0810 11:34:46.386851 (+ 682us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":583,"thread_start_us":363,"threads_started":1}
I20260810 11:34:46.391003 25096 heartbeater.cc:347] Connected to a master server at 127.8.18.254:42587
I20260810 11:34:46.391449 25096 heartbeater.cc:464] Registering TS with master...
I20260810 11:34:46.392690 25096 heartbeater.cc:511] Master 127.8.18.254:42587 requested a full tablet report, sending...
I20260810 11:34:46.395089 24775 ts_manager.cc:194] Registered new tserver with Master: 1103ed0eb8974fd8bc3eb2955eacbc78 (127.8.18.194:34511)
I20260810 11:34:46.396567 24775 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.18.194@KRBTEST.COM'} at 127.8.18.194:54793
W20260810 11:34:46.749576 25100 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:34:46.749986 25100 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:34:46.750162 25100 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:34:46.758422 25100 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:34:46.758642 25100 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:34:46.758939 25100 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.18.195
I20260810 11:34:46.774009 25100 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.18.212:33987
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.195.keytab
--principal=kudu/127.8.18.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.18.195:40573
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.18.195
--webserver_port=46223
--tserver_master_addrs=127.8.18.254:42587
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.18.195
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:34:46.775449 25100 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:34:46.776847 25100 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:34:46.791433 25105 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:34:46.791720 25106 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:34:46.791486 25108 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:34:46.792943 25100 server_base.cc:1034] running on GCE node
I20260810 11:34:46.793829 25100 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:34:46.794953 25100 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:34:46.796234 25100 hybrid_clock.cc:648] HybridClock initialized: now 1786361686796162 us; error 63 us; skew 500 ppm
I20260810 11:34:46.796623 25100 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:34:46 dist-test-slave-srgv krb5kdc[24177](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361686, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:34:46.800465 25100 init.cc:377] Logged in from keytab as kudu/127.8.18.195@KRBTEST.COM (short username kudu)
I20260810 11:34:46.802552 25100 webserver.cc:460] Webserver started at http://127.8.18.195:46223/ using document root <none> and password file <none>
I20260810 11:34:46.803390 25100 fs_manager.cc:359] Metadata directory not provided
I20260810 11:34:46.803591 25100 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:34:46.810308 25100 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260810 11:34:46.813714 25116 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:34:46.815533 25100 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260810 11:34:46.815810 25100 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/ts-2/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/ts-2/wal
uuid: "f87e487a91b542408b4a24ffc52421be"
format_stamp: "Formatted at 2026-08-10 11:34:23 on dist-test-slave-srgv"
server_key: "f937034459273ddb79f12ef371db5203"
server_key_iv: "c5654e1e7bfbd521d2210a671cc5a52a"
server_key_version: "encryptionkey@0"
I20260810 11:34:46.816991 25100 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:34:46.834909 25100 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:34:46.840077 25100 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:34:46.840408 25100 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:34:46.841573 25100 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:34:46.843129 25100 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:34:46.843269 25100 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:34:46.843412 25100 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:34:46.843508 25100 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:34:46.873486 24962 heartbeater.cc:503] Master 127.8.18.254:42587 was elected leader, sending a full tablet report...
I20260810 11:34:46.880946 25100 rpc_server.cc:307] RPC server started. Bound to: 127.8.18.195:40573
I20260810 11:34:46.881050 25229 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.18.195:40573 every 8 connection(s)
I20260810 11:34:46.882658 25100 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/ts-2/data/info.pb
I20260810 11:34:46.883101 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 25100
Aug 10 11:34:46 dist-test-slave-srgv krb5kdc[24177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361686, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.195@KRBTEST.COM for kudu/127.8.18.254@KRBTEST.COM
I20260810 11:34:46.902540 25098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:34:46.884299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49610 (local address 127.8.18.254:42587)
0810 11:34:46.884487 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0810 11:34:46.884493 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:34:46.885108 (+ 615us) server_negotiation.cc:407] Connection header received
0810 11:34:46.885332 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:34:46.885339 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:34:46.885487 (+ 148us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:34:46.885608 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:34:46.887673 (+ 2065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:34:46.888841 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:34:46.891164 (+ 2323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:34:46.891820 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:34:46.891925 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:34:46.895438 (+ 3513us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:34:46.895444 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:34:46.895447 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:34:46.895488 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:34:46.899052 (+ 3564us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:34:46.900096 (+ 1044us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:34:46.900103 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:34:46.900107 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:34:46.900255 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:34:46.900747 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:34:46.900760 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:34:46.900763 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:34:46.901164 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:34:46.901516 (+ 352us) server_negotiation.cc:1035] Waiting for connection context
0810 11:34:46.901875 (+ 359us) server_negotiation.cc:299] Negotiation successful
0810 11:34:46.902166 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260810 11:34:46.909799 25233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:34:46.885869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.195:47015 (local address 127.8.18.254:42587)
0810 11:34:46.887551 (+ 1682us) server_negotiation.cc:206] Beginning negotiation
0810 11:34:46.887560 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 11:34:46.888296 (+ 736us) server_negotiation.cc:407] Connection header received
0810 11:34:46.891164 (+ 2868us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:34:46.891171 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:34:46.891297 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:34:46.891426 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:34:46.894070 (+ 2644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:34:46.894892 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:34:46.896842 (+ 1950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:34:46.897683 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:34:46.897845 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:34:46.902142 (+ 4297us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:34:46.902148 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:34:46.902152 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:34:46.902184 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:34:46.906081 (+ 3897us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:34:46.907004 (+ 923us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:34:46.907012 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:34:46.907021 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:34:46.907156 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:34:46.907689 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:34:46.907693 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:34:46.907695 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:34:46.908079 (+ 384us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:34:46.908254 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0810 11:34:46.909049 (+ 795us) server_negotiation.cc:299] Negotiation successful
0810 11:34:46.909331 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1538,"thread_start_us":1404,"threads_started":1}
I20260810 11:34:46.910823 25234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:34:46.886848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:42587 (local address 127.8.18.195:47015)
0810 11:34:46.888117 (+ 1269us) negotiation.cc:107] Waiting for socket to connect
0810 11:34:46.888162 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0810 11:34:46.890742 (+ 2580us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:34:46.891696 (+ 954us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:34:46.891707 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:34:46.892758 (+ 1051us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:34:46.893812 (+ 1054us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:34:46.893848 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:34:46.895103 (+ 1255us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:34:46.895110 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:34:46.896645 (+ 1535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:34:46.896664 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:34:46.897946 (+ 1282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:34:46.897952 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:34:46.898187 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:34:46.898879 (+ 692us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:34:46.898905 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:34:46.901842 (+ 2937us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:34:46.906295 (+ 4453us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:34:46.906307 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:34:46.906334 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:34:46.906844 (+ 510us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:34:46.907285 (+ 441us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:34:46.907290 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:34:46.907294 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:34:46.907534 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:34:46.908229 (+ 695us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:34:46.908238 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:34:46.908647 (+ 409us) client_negotiation.cc:769] Sending connection context
0810 11:34:46.908940 (+ 293us) client_negotiation.cc:240] Negotiation successful
0810 11:34:46.909596 (+ 656us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":969,"thread_start_us":714,"threads_started":1}
I20260810 11:34:46.912492 25230 heartbeater.cc:347] Connected to a master server at 127.8.18.254:42587
I20260810 11:34:46.912904 25230 heartbeater.cc:464] Registering TS with master...
I20260810 11:34:46.913832 25230 heartbeater.cc:511] Master 127.8.18.254:42587 requested a full tablet report, sending...
I20260810 11:34:46.915520 24775 ts_manager.cc:194] Registered new tserver with Master: f87e487a91b542408b4a24ffc52421be (127.8.18.195:40573)
I20260810 11:34:46.916342 24775 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.18.195@KRBTEST.COM'} at 127.8.18.195:47015
I20260810 11:34:46.918777 8267 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 11:34:46.926831 25233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:34:46.919935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49616 (local address 127.8.18.254:42587)
0810 11:34:46.920153 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0810 11:34:46.920162 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 11:34:46.920595 (+ 433us) server_negotiation.cc:407] Connection header received
0810 11:34:46.920873 (+ 278us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:34:46.920917 (+ 44us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:34:46.921094 (+ 177us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:34:46.921307 (+ 213us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 11:34:46.921889 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:34:46.922783 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:34:46.924411 (+ 1628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:34:46.924874 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:34:46.925080 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:34:46.925727 (+ 647us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 11:34:46.926014 (+ 287us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0810 11:34:46.926505 (+ 491us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:49616: token expired
Metrics: {"server-negotiator.queue_time_us":57}
W20260810 11:34:46.926995 25233 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:49616: token expired
W20260810 11:34:46.927490 24695 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.8.18.254:42587)
I20260810 11:34:46.943290 25233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:34:46.928204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49618 (local address 127.8.18.254:42587)
0810 11:34:46.928388 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0810 11:34:46.928392 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 11:34:46.928408 (+ 16us) server_negotiation.cc:407] Connection header received
0810 11:34:46.928453 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:34:46.928456 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:34:46.928514 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:34:46.928611 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:34:46.930238 (+ 1627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:34:46.930985 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:34:46.932510 (+ 1525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:34:46.932933 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:34:46.933078 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:34:46.936194 (+ 3116us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:34:46.936201 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:34:46.936204 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:34:46.936249 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:34:46.939762 (+ 3513us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:34:46.940790 (+ 1028us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:34:46.940797 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:34:46.940808 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:34:46.940967 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:34:46.941476 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:34:46.941480 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:34:46.941482 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:34:46.941869 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:34:46.942237 (+ 368us) server_negotiation.cc:1035] Waiting for connection context
0810 11:34:46.942580 (+ 343us) server_negotiation.cc:299] Negotiation successful
0810 11:34:46.942866 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260810 11:34:46.947865 24775 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49618:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 11:34:46.956164 24775 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 11:34:47.001561 25245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:34:46.971947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54622 (local address 127.8.18.193:43511)
0810 11:34:46.990056 (+ 18109us) server_negotiation.cc:206] Beginning negotiation
0810 11:34:46.990068 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 11:34:46.992841 (+ 2773us) server_negotiation.cc:407] Connection header received
0810 11:34:46.993014 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:34:46.993020 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:34:46.993197 (+ 177us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:34:46.993399 (+ 202us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:34:46.994164 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:34:46.995183 (+ 1019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:34:46.999017 (+ 3834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:34:47.000068 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:34:47.000265 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:34:47.000552 (+ 287us) server_negotiation.cc:1035] Waiting for connection context
0810 11:34:47.000937 (+ 385us) server_negotiation.cc:299] Negotiation successful
0810 11:34:47.001131 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17889,"thread_start_us":17709,"threads_started":1}
I20260810 11:34:47.001668 25240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:34:46.971091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.193:43511 (local address 127.0.0.1:54622)
0810 11:34:46.992633 (+ 21542us) negotiation.cc:107] Waiting for socket to connect
0810 11:34:46.992662 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0810 11:34:46.992849 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:34:46.993514 (+ 665us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:34:46.993524 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:34:46.993554 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:34:46.993978 (+ 424us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:34:46.993989 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:34:46.995339 (+ 1350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:34:46.995344 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:34:46.998833 (+ 3489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:34:46.998844 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:34:47.000359 (+ 1515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:34:47.000366 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:34:47.000660 (+ 294us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:34:47.000710 (+ 50us) client_negotiation.cc:769] Sending connection context
0810 11:34:47.000870 (+ 160us) client_negotiation.cc:240] Negotiation successful
0810 11:34:47.001117 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21402,"mutex_wait_us":249,"thread_start_us":19517,"threads_started":1}
I20260810 11:34:47.002095 25244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:34:46.971428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36482 (local address 127.8.18.195:40573)
0810 11:34:46.989077 (+ 17649us) server_negotiation.cc:206] Beginning negotiation
0810 11:34:46.989092 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 11:34:46.991498 (+ 2406us) server_negotiation.cc:407] Connection header received
0810 11:34:46.991621 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:34:46.991628 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:34:46.991845 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:34:46.992030 (+ 185us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:34:46.993628 (+ 1598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:34:46.995382 (+ 1754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:34:46.998647 (+ 3265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:34:47.000289 (+ 1642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:34:47.000433 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:34:47.000654 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0810 11:34:47.001551 (+ 897us) server_negotiation.cc:299] Negotiation successful
0810 11:34:47.001733 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17368,"thread_start_us":17676,"threads_started":1}
I20260810 11:34:47.002208 25242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:34:46.970374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.194:34511 (local address 127.0.0.1:51476)
0810 11:34:46.990420 (+ 20046us) negotiation.cc:107] Waiting for socket to connect
0810 11:34:46.990483 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0810 11:34:46.990944 (+ 461us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:34:46.992238 (+ 1294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:34:46.992246 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:34:46.992297 (+ 51us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:34:46.993322 (+ 1025us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:34:46.993337 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:34:46.995688 (+ 2351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:34:46.995692 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:34:46.998365 (+ 2673us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:34:46.998383 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:34:47.000965 (+ 2582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:34:47.000971 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:34:47.001252 (+ 281us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:34:47.001323 (+ 71us) client_negotiation.cc:769] Sending connection context
0810 11:34:47.001400 (+ 77us) client_negotiation.cc:240] Negotiation successful
0810 11:34:47.001597 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19447,"mutex_wait_us":448,"spinlock_wait_cycles":2304,"thread_start_us":21789,"threads_started":1}
I20260810 11:34:47.002490 25241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:34:46.970524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51476 (local address 127.8.18.194:34511)
0810 11:34:46.985984 (+ 15460us) server_negotiation.cc:206] Beginning negotiation
0810 11:34:46.985998 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 11:34:46.990675 (+ 4677us) server_negotiation.cc:407] Connection header received
0810 11:34:46.991322 (+ 647us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:34:46.991331 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:34:46.991636 (+ 305us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:34:46.991870 (+ 234us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:34:46.993636 (+ 1766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:34:46.995432 (+ 1796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:34:46.998666 (+ 3234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:34:47.000280 (+ 1614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:34:47.000412 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:34:47.000628 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0810 11:34:47.001794 (+ 1166us) server_negotiation.cc:299] Negotiation successful
0810 11:34:47.001985 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15195,"thread_start_us":15481,"threads_started":1}
I20260810 11:34:47.002992 25243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:34:46.970909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.195:40573 (local address 127.0.0.1:36482)
0810 11:34:46.991166 (+ 20257us) negotiation.cc:107] Waiting for socket to connect
0810 11:34:46.991193 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0810 11:34:46.991361 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:34:46.992405 (+ 1044us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:34:46.992410 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:34:46.992443 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:34:46.993322 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:34:46.993337 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:34:46.995548 (+ 2211us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:34:46.995553 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:34:46.998433 (+ 2880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:34:46.998450 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:34:47.000952 (+ 2502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:34:47.000960 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:34:47.001253 (+ 293us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:34:47.001318 (+ 65us) client_negotiation.cc:769] Sending connection context
0810 11:34:47.001400 (+ 82us) client_negotiation.cc:240] Negotiation successful
0810 11:34:47.002496 (+ 1096us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20075,"mutex_wait_us":259,"spinlock_wait_cycles":7296,"thread_start_us":18726,"threads_started":1}
I20260810 11:34:47.006918 25030 tablet_service.cc:1467] Processing CreateTablet for tablet 16cc68ea874b48f19573f4e739edcc74 (DEFAULT_TABLE table=test-table [id=f905ef31a5434d799b04b887518d4e29]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:34:47.007012 25164 tablet_service.cc:1467] Processing CreateTablet for tablet 16cc68ea874b48f19573f4e739edcc74 (DEFAULT_TABLE table=test-table [id=f905ef31a5434d799b04b887518d4e29]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:34:47.008333 25030 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 16cc68ea874b48f19573f4e739edcc74. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:34:47.007931 24896 tablet_service.cc:1467] Processing CreateTablet for tablet 16cc68ea874b48f19573f4e739edcc74 (DEFAULT_TABLE table=test-table [id=f905ef31a5434d799b04b887518d4e29]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:34:47.008534 25164 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 16cc68ea874b48f19573f4e739edcc74. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:34:47.009536 24896 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 16cc68ea874b48f19573f4e739edcc74. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:34:47.037930 25248 tablet_bootstrap.cc:492] T 16cc68ea874b48f19573f4e739edcc74 P 1103ed0eb8974fd8bc3eb2955eacbc78: Bootstrap starting.
I20260810 11:34:47.045737 25248 tablet_bootstrap.cc:654] T 16cc68ea874b48f19573f4e739edcc74 P 1103ed0eb8974fd8bc3eb2955eacbc78: Neither blocks nor log segments found. Creating new log.
I20260810 11:34:47.046904 25246 tablet_bootstrap.cc:492] T 16cc68ea874b48f19573f4e739edcc74 P 8a82dd1f6f654360b68d7ff014fe5e71: Bootstrap starting.
I20260810 11:34:47.049890 25247 tablet_bootstrap.cc:492] T 16cc68ea874b48f19573f4e739edcc74 P f87e487a91b542408b4a24ffc52421be: Bootstrap starting.
I20260810 11:34:47.052224 25246 tablet_bootstrap.cc:654] T 16cc68ea874b48f19573f4e739edcc74 P 8a82dd1f6f654360b68d7ff014fe5e71: Neither blocks nor log segments found. Creating new log.
I20260810 11:34:47.053925 25247 tablet_bootstrap.cc:654] T 16cc68ea874b48f19573f4e739edcc74 P f87e487a91b542408b4a24ffc52421be: Neither blocks nor log segments found. Creating new log.
I20260810 11:34:47.054065 25246 log.cc:824] T 16cc68ea874b48f19573f4e739edcc74 P 8a82dd1f6f654360b68d7ff014fe5e71: Log is configured to *not* fsync() on all Append() calls
I20260810 11:34:47.055076 25248 log.cc:824] T 16cc68ea874b48f19573f4e739edcc74 P 1103ed0eb8974fd8bc3eb2955eacbc78: Log is configured to *not* fsync() on all Append() calls
I20260810 11:34:47.055311 25247 log.cc:824] T 16cc68ea874b48f19573f4e739edcc74 P f87e487a91b542408b4a24ffc52421be: Log is configured to *not* fsync() on all Append() calls
I20260810 11:34:47.063218 25247 tablet_bootstrap.cc:492] T 16cc68ea874b48f19573f4e739edcc74 P f87e487a91b542408b4a24ffc52421be: No bootstrap required, opened a new log
I20260810 11:34:47.063774 25247 ts_tablet_manager.cc:1397] T 16cc68ea874b48f19573f4e739edcc74 P f87e487a91b542408b4a24ffc52421be: Time spent bootstrapping tablet: real 0.014s user 0.009s sys 0.003s
I20260810 11:34:47.064383 25246 tablet_bootstrap.cc:492] T 16cc68ea874b48f19573f4e739edcc74 P 8a82dd1f6f654360b68d7ff014fe5e71: No bootstrap required, opened a new log
I20260810 11:34:47.064863 25246 ts_tablet_manager.cc:1397] T 16cc68ea874b48f19573f4e739edcc74 P 8a82dd1f6f654360b68d7ff014fe5e71: Time spent bootstrapping tablet: real 0.018s user 0.012s sys 0.005s
I20260810 11:34:47.071280 25248 tablet_bootstrap.cc:492] T 16cc68ea874b48f19573f4e739edcc74 P 1103ed0eb8974fd8bc3eb2955eacbc78: No bootstrap required, opened a new log
I20260810 11:34:47.071738 25248 ts_tablet_manager.cc:1397] T 16cc68ea874b48f19573f4e739edcc74 P 1103ed0eb8974fd8bc3eb2955eacbc78: Time spent bootstrapping tablet: real 0.034s user 0.003s sys 0.019s
I20260810 11:34:47.078542 25247 raft_consensus.cc:348] T 16cc68ea874b48f19573f4e739edcc74 P f87e487a91b542408b4a24ffc52421be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1103ed0eb8974fd8bc3eb2955eacbc78" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 34511 } } peers { permanent_uuid: "8a82dd1f6f654360b68d7ff014fe5e71" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 43511 } } peers { permanent_uuid: "f87e487a91b542408b4a24ffc52421be" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 40573 } }
I20260810 11:34:47.079320 25247 raft_consensus.cc:374] T 16cc68ea874b48f19573f4e739edcc74 P f87e487a91b542408b4a24ffc52421be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:34:47.079504 25247 raft_consensus.cc:729] T 16cc68ea874b48f19573f4e739edcc74 P f87e487a91b542408b4a24ffc52421be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f87e487a91b542408b4a24ffc52421be, State: Initialized, Role: FOLLOWER
I20260810 11:34:47.079246 25246 raft_consensus.cc:348] T 16cc68ea874b48f19573f4e739edcc74 P 8a82dd1f6f654360b68d7ff014fe5e71 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1103ed0eb8974fd8bc3eb2955eacbc78" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 34511 } } peers { permanent_uuid: "8a82dd1f6f654360b68d7ff014fe5e71" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 43511 } } peers { permanent_uuid: "f87e487a91b542408b4a24ffc52421be" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 40573 } }
I20260810 11:34:47.079958 25246 raft_consensus.cc:374] T 16cc68ea874b48f19573f4e739edcc74 P 8a82dd1f6f654360b68d7ff014fe5e71 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:34:47.080132 25246 raft_consensus.cc:729] T 16cc68ea874b48f19573f4e739edcc74 P 8a82dd1f6f654360b68d7ff014fe5e71 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a82dd1f6f654360b68d7ff014fe5e71, State: Initialized, Role: FOLLOWER
I20260810 11:34:47.080473 25247 consensus_queue.cc:260] T 16cc68ea874b48f19573f4e739edcc74 P f87e487a91b542408b4a24ffc52421be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1103ed0eb8974fd8bc3eb2955eacbc78" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 34511 } } peers { permanent_uuid: "8a82dd1f6f654360b68d7ff014fe5e71" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 43511 } } peers { permanent_uuid: "f87e487a91b542408b4a24ffc52421be" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 40573 } }
I20260810 11:34:47.080938 25246 consensus_queue.cc:260] T 16cc68ea874b48f19573f4e739edcc74 P 8a82dd1f6f654360b68d7ff014fe5e71 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1103ed0eb8974fd8bc3eb2955eacbc78" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 34511 } } peers { permanent_uuid: "8a82dd1f6f654360b68d7ff014fe5e71" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 43511 } } peers { permanent_uuid: "f87e487a91b542408b4a24ffc52421be" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 40573 } }
I20260810 11:34:47.086177 25248 raft_consensus.cc:348] T 16cc68ea874b48f19573f4e739edcc74 P 1103ed0eb8974fd8bc3eb2955eacbc78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1103ed0eb8974fd8bc3eb2955eacbc78" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 34511 } } peers { permanent_uuid: "8a82dd1f6f654360b68d7ff014fe5e71" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 43511 } } peers { permanent_uuid: "f87e487a91b542408b4a24ffc52421be" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 40573 } }
I20260810 11:34:47.087074 25248 raft_consensus.cc:374] T 16cc68ea874b48f19573f4e739edcc74 P 1103ed0eb8974fd8bc3eb2955eacbc78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:34:47.087271 25248 raft_consensus.cc:729] T 16cc68ea874b48f19573f4e739edcc74 P 1103ed0eb8974fd8bc3eb2955eacbc78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1103ed0eb8974fd8bc3eb2955eacbc78, State: Initialized, Role: FOLLOWER
I20260810 11:34:47.088055 25230 heartbeater.cc:503] Master 127.8.18.254:42587 was elected leader, sending a full tablet report...
I20260810 11:34:47.089200 25247 ts_tablet_manager.cc:1428] T 16cc68ea874b48f19573f4e739edcc74 P f87e487a91b542408b4a24ffc52421be: Time spent starting tablet: real 0.025s user 0.018s sys 0.004s
I20260810 11:34:47.089666 25246 ts_tablet_manager.cc:1428] T 16cc68ea874b48f19573f4e739edcc74 P 8a82dd1f6f654360b68d7ff014fe5e71: Time spent starting tablet: real 0.025s user 0.016s sys 0.007s
I20260810 11:34:47.090075 25248 consensus_queue.cc:260] T 16cc68ea874b48f19573f4e739edcc74 P 1103ed0eb8974fd8bc3eb2955eacbc78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1103ed0eb8974fd8bc3eb2955eacbc78" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 34511 } } peers { permanent_uuid: "8a82dd1f6f654360b68d7ff014fe5e71" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 43511 } } peers { permanent_uuid: "f87e487a91b542408b4a24ffc52421be" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 40573 } }
I20260810 11:34:47.093308 25253 raft_consensus.cc:482] T 16cc68ea874b48f19573f4e739edcc74 P 8a82dd1f6f654360b68d7ff014fe5e71 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 11:34:47.093786 25253 raft_consensus.cc:504] T 16cc68ea874b48f19573f4e739edcc74 P 8a82dd1f6f654360b68d7ff014fe5e71 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1103ed0eb8974fd8bc3eb2955eacbc78" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 34511 } } peers { permanent_uuid: "8a82dd1f6f654360b68d7ff014fe5e71" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 43511 } } peers { permanent_uuid: "f87e487a91b542408b4a24ffc52421be" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 40573 } }
I20260810 11:34:47.102943 25253 leader_election.cc:288] T 16cc68ea874b48f19573f4e739edcc74 P 8a82dd1f6f654360b68d7ff014fe5e71 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1103ed0eb8974fd8bc3eb2955eacbc78 (127.8.18.194:34511), f87e487a91b542408b4a24ffc52421be (127.8.18.195:40573)
I20260810 11:34:47.106513 25096 heartbeater.cc:503] Master 127.8.18.254:42587 was elected leader, sending a full tablet report...
I20260810 11:34:47.107595 25248 ts_tablet_manager.cc:1428] T 16cc68ea874b48f19573f4e739edcc74 P 1103ed0eb8974fd8bc3eb2955eacbc78: Time spent starting tablet: real 0.036s user 0.015s sys 0.017s
W20260810 11:34:47.112458 25097 tablet.cc:2369] T 16cc68ea874b48f19573f4e739edcc74 P 1103ed0eb8974fd8bc3eb2955eacbc78: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 11:34:47.117460 25244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:34:47.097570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.193:40421 (local address 127.8.18.195:40573)
0810 11:34:47.098394 (+ 824us) server_negotiation.cc:206] Beginning negotiation
0810 11:34:47.098404 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 11:34:47.108779 (+ 10375us) server_negotiation.cc:407] Connection header received
0810 11:34:47.109108 (+ 329us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:34:47.109114 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:34:47.109196 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:34:47.109267 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:34:47.110248 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:34:47.111587 (+ 1339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:34:47.114430 (+ 2843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:34:47.115890 (+ 1460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:34:47.116090 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:34:47.116205 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0810 11:34:47.116966 (+ 761us) server_negotiation.cc:299] Negotiation successful
0810 11:34:47.117161 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":594}
I20260810 11:34:47.117595 25241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:34:47.098758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.193:58831 (local address 127.8.18.194:34511)
0810 11:34:47.099040 (+ 282us) server_negotiation.cc:206] Beginning negotiation
0810 11:34:47.099048 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:34:47.109729 (+ 10681us) server_negotiation.cc:407] Connection header received
0810 11:34:47.109889 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:34:47.109897 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:34:47.109985 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:34:47.110088 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:34:47.110870 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:34:47.111999 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:34:47.115216 (+ 3217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:34:47.116405 (+ 1189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:34:47.116627 (+ 222us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:34:47.116712 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0810 11:34:47.117153 (+ 441us) server_negotiation.cc:299] Negotiation successful
0810 11:34:47.117292 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260810 11:34:47.117688 25256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:34:47.096807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.195:40573 (local address 127.8.18.193:40421)
0810 11:34:47.108627 (+ 11820us) negotiation.cc:107] Waiting for socket to connect
0810 11:34:47.108652 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0810 11:34:47.108925 (+ 273us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:34:47.109560 (+ 635us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:34:47.109566 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:34:47.109595 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:34:47.110048 (+ 453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:34:47.110067 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:34:47.111814 (+ 1747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:34:47.111820 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:34:47.114220 (+ 2400us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:34:47.114237 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:34:47.116394 (+ 2157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:34:47.116404 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:34:47.116660 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:34:47.116779 (+ 119us) client_negotiation.cc:769] Sending connection context
0810 11:34:47.116885 (+ 106us) client_negotiation.cc:240] Negotiation successful
0810 11:34:47.117089 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11638,"mutex_wait_us":29,"thread_start_us":12463,"threads_started":1}
I20260810 11:34:47.118363 25255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:34:47.097748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.194:34511 (local address 127.8.18.193:58831)
0810 11:34:47.109623 (+ 11875us) negotiation.cc:107] Waiting for socket to connect
0810 11:34:47.109639 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 11:34:47.109776 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:34:47.110251 (+ 475us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:34:47.110256 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:34:47.110281 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:34:47.110673 (+ 392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:34:47.110686 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:34:47.112901 (+ 2215us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:34:47.112913 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:34:47.115013 (+ 2100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:34:47.115030 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:34:47.116657 (+ 1627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:34:47.116664 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:34:47.116893 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:34:47.116935 (+ 42us) client_negotiation.cc:769] Sending connection context
0810 11:34:47.117859 (+ 924us) client_negotiation.cc:240] Negotiation successful
0810 11:34:47.118016 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11737,"mutex_wait_us":31,"thread_start_us":10470,"threads_started":1}
I20260810 11:34:47.119019 25184 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16cc68ea874b48f19573f4e739edcc74" candidate_uuid: "8a82dd1f6f654360b68d7ff014fe5e71" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f87e487a91b542408b4a24ffc52421be" is_pre_election: true
I20260810 11:34:47.119475 25050 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16cc68ea874b48f19573f4e739edcc74" candidate_uuid: "8a82dd1f6f654360b68d7ff014fe5e71" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1103ed0eb8974fd8bc3eb2955eacbc78" is_pre_election: true
I20260810 11:34:47.119689 25184 raft_consensus.cc:2454] T 16cc68ea874b48f19573f4e739edcc74 P f87e487a91b542408b4a24ffc52421be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a82dd1f6f654360b68d7ff014fe5e71 in term 0.
I20260810 11:34:47.120021 25050 raft_consensus.cc:2454] T 16cc68ea874b48f19573f4e739edcc74 P 1103ed0eb8974fd8bc3eb2955eacbc78 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a82dd1f6f654360b68d7ff014fe5e71 in term 0.
I20260810 11:34:47.120755 24852 leader_election.cc:302] T 16cc68ea874b48f19573f4e739edcc74 P 8a82dd1f6f654360b68d7ff014fe5e71 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a82dd1f6f654360b68d7ff014fe5e71, f87e487a91b542408b4a24ffc52421be; no voters:
I20260810 11:34:47.121290 25253 raft_consensus.cc:2781] T 16cc68ea874b48f19573f4e739edcc74 P 8a82dd1f6f654360b68d7ff014fe5e71 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 11:34:47.121533 25253 raft_consensus.cc:482] T 16cc68ea874b48f19573f4e739edcc74 P 8a82dd1f6f654360b68d7ff014fe5e71 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 11:34:47.121663 25253 raft_consensus.cc:3037] T 16cc68ea874b48f19573f4e739edcc74 P 8a82dd1f6f654360b68d7ff014fe5e71 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:34:47.124399 25253 raft_consensus.cc:504] T 16cc68ea874b48f19573f4e739edcc74 P 8a82dd1f6f654360b68d7ff014fe5e71 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1103ed0eb8974fd8bc3eb2955eacbc78" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 34511 } } peers { permanent_uuid: "8a82dd1f6f654360b68d7ff014fe5e71" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 43511 } } peers { permanent_uuid: "f87e487a91b542408b4a24ffc52421be" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 40573 } }
I20260810 11:34:47.125254 25253 leader_election.cc:288] T 16cc68ea874b48f19573f4e739edcc74 P 8a82dd1f6f654360b68d7ff014fe5e71 [CANDIDATE]: Term 1 election: Requested vote from peers 1103ed0eb8974fd8bc3eb2955eacbc78 (127.8.18.194:34511), f87e487a91b542408b4a24ffc52421be (127.8.18.195:40573)
I20260810 11:34:47.125964 25050 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16cc68ea874b48f19573f4e739edcc74" candidate_uuid: "8a82dd1f6f654360b68d7ff014fe5e71" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1103ed0eb8974fd8bc3eb2955eacbc78"
I20260810 11:34:47.126044 25184 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16cc68ea874b48f19573f4e739edcc74" candidate_uuid: "8a82dd1f6f654360b68d7ff014fe5e71" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f87e487a91b542408b4a24ffc52421be"
I20260810 11:34:47.126299 25050 raft_consensus.cc:3037] T 16cc68ea874b48f19573f4e739edcc74 P 1103ed0eb8974fd8bc3eb2955eacbc78 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:34:47.126298 25184 raft_consensus.cc:3037] T 16cc68ea874b48f19573f4e739edcc74 P f87e487a91b542408b4a24ffc52421be [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:34:47.128906 25184 raft_consensus.cc:2454] T 16cc68ea874b48f19573f4e739edcc74 P f87e487a91b542408b4a24ffc52421be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a82dd1f6f654360b68d7ff014fe5e71 in term 1.
I20260810 11:34:47.129621 24852 leader_election.cc:302] T 16cc68ea874b48f19573f4e739edcc74 P 8a82dd1f6f654360b68d7ff014fe5e71 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a82dd1f6f654360b68d7ff014fe5e71, f87e487a91b542408b4a24ffc52421be; no voters:
I20260810 11:34:47.130059 25253 raft_consensus.cc:2781] T 16cc68ea874b48f19573f4e739edcc74 P 8a82dd1f6f654360b68d7ff014fe5e71 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:34:47.130844 25050 raft_consensus.cc:2454] T 16cc68ea874b48f19573f4e739edcc74 P 1103ed0eb8974fd8bc3eb2955eacbc78 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a82dd1f6f654360b68d7ff014fe5e71 in term 1.
I20260810 11:34:47.130810 25253 raft_consensus.cc:686] T 16cc68ea874b48f19573f4e739edcc74 P 8a82dd1f6f654360b68d7ff014fe5e71 [term 1 LEADER]: Becoming Leader. State: Replica: 8a82dd1f6f654360b68d7ff014fe5e71, State: Running, Role: LEADER
I20260810 11:34:47.131922 25253 consensus_queue.cc:237] T 16cc68ea874b48f19573f4e739edcc74 P 8a82dd1f6f654360b68d7ff014fe5e71 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1103ed0eb8974fd8bc3eb2955eacbc78" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 34511 } } peers { permanent_uuid: "8a82dd1f6f654360b68d7ff014fe5e71" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 43511 } } peers { permanent_uuid: "f87e487a91b542408b4a24ffc52421be" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 40573 } }
W20260810 11:34:47.135147 25231 tablet.cc:2369] T 16cc68ea874b48f19573f4e739edcc74 P f87e487a91b542408b4a24ffc52421be: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 11:34:47.137844 24775 catalog_manager.cc:5495] T 16cc68ea874b48f19573f4e739edcc74 P 8a82dd1f6f654360b68d7ff014fe5e71 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8a82dd1f6f654360b68d7ff014fe5e71 (127.8.18.193). New cstate: current_term: 1 leader_uuid: "8a82dd1f6f654360b68d7ff014fe5e71" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1103ed0eb8974fd8bc3eb2955eacbc78" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 34511 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a82dd1f6f654360b68d7ff014fe5e71" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 43511 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f87e487a91b542408b4a24ffc52421be" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 40573 } health_report { overall_health: UNKNOWN } } }
I20260810 11:34:47.158955 25245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:34:47.152896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54634 (local address 127.8.18.193:43511)
0810 11:34:47.153145 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0810 11:34:47.153152 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:34:47.153297 (+ 145us) server_negotiation.cc:407] Connection header received
0810 11:34:47.153666 (+ 369us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:34:47.153670 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:34:47.153765 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:34:47.153936 (+ 171us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 11:34:47.154396 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:34:47.155168 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:34:47.156596 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:34:47.157012 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:34:47.157159 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:34:47.157784 (+ 625us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 11:34:47.158317 (+ 533us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 11:34:47.158502 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0810 11:34:47.158571 (+ 69us) server_negotiation.cc:299] Negotiation successful
0810 11:34:47.158668 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260810 11:34:47.169898 25184 raft_consensus.cc:1261] T 16cc68ea874b48f19573f4e739edcc74 P f87e487a91b542408b4a24ffc52421be [term 1 FOLLOWER]: Refusing update from remote peer 8a82dd1f6f654360b68d7ff014fe5e71: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 11:34:47.170046 25050 raft_consensus.cc:1261] T 16cc68ea874b48f19573f4e739edcc74 P 1103ed0eb8974fd8bc3eb2955eacbc78 [term 1 FOLLOWER]: Refusing update from remote peer 8a82dd1f6f654360b68d7ff014fe5e71: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 11:34:47.171322 25257 consensus_queue.cc:1035] T 16cc68ea874b48f19573f4e739edcc74 P 8a82dd1f6f654360b68d7ff014fe5e71 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1103ed0eb8974fd8bc3eb2955eacbc78" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 34511 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 11:34:47.172242 25253 consensus_queue.cc:1035] T 16cc68ea874b48f19573f4e739edcc74 P 8a82dd1f6f654360b68d7ff014fe5e71 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f87e487a91b542408b4a24ffc52421be" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 40573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 11:34:47.187708 25260 mvcc.cc:205] Tried to move back new op lower bound from 7316937470634508288 to 7316937470496485376. Current Snapshot: MvccSnapshot[applied={T|T < 7316937470634508288}]
I20260810 11:34:47.210093 25262 mvcc.cc:205] Tried to move back new op lower bound from 7316937470634508288 to 7316937470496485376. Current Snapshot: MvccSnapshot[applied={T|T < 7316937470634508288}]
I20260810 11:34:47.222195 24774 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49618:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 11:34:47.222750 24774 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49618:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 11:34:47.228880 24774 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 25ef7a60224c4f7bb5147d7c5ed8251c: Sending DeleteTablet for 3 replicas of tablet 16cc68ea874b48f19573f4e739edcc74
I20260810 11:34:47.230443 24896 tablet_service.cc:1514] Processing DeleteTablet for tablet 16cc68ea874b48f19573f4e739edcc74 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:34:47 UTC) from {username='kudu', principal='kudu/127.8.18.254@KRBTEST.COM'} at 127.0.0.1:54622
I20260810 11:34:47.230561 25030 tablet_service.cc:1514] Processing DeleteTablet for tablet 16cc68ea874b48f19573f4e739edcc74 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:34:47 UTC) from {username='kudu', principal='kudu/127.8.18.254@KRBTEST.COM'} at 127.0.0.1:51476
I20260810 11:34:47.234532 25164 tablet_service.cc:1514] Processing DeleteTablet for tablet 16cc68ea874b48f19573f4e739edcc74 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:34:47 UTC) from {username='kudu', principal='kudu/127.8.18.254@KRBTEST.COM'} at 127.0.0.1:36482
I20260810 11:34:47.235589 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 24813
I20260810 11:34:47.249454 25271 tablet_replica.cc:330] T 16cc68ea874b48f19573f4e739edcc74 P 8a82dd1f6f654360b68d7ff014fe5e71: stopping tablet replica
I20260810 11:34:47.250061 25271 raft_consensus.cc:2229] T 16cc68ea874b48f19573f4e739edcc74 P 8a82dd1f6f654360b68d7ff014fe5e71 [term 1 LEADER]: Raft consensus shutting down.
I20260810 11:34:47.250867 25271 raft_consensus.cc:2258] T 16cc68ea874b48f19573f4e739edcc74 P 8a82dd1f6f654360b68d7ff014fe5e71 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:34:47.253720 25271 ts_tablet_manager.cc:1905] T 16cc68ea874b48f19573f4e739edcc74 P 8a82dd1f6f654360b68d7ff014fe5e71: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:34:47.260185 25272 tablet_replica.cc:330] T 16cc68ea874b48f19573f4e739edcc74 P 1103ed0eb8974fd8bc3eb2955eacbc78: stopping tablet replica
I20260810 11:34:47.260711 25272 raft_consensus.cc:2229] T 16cc68ea874b48f19573f4e739edcc74 P 1103ed0eb8974fd8bc3eb2955eacbc78 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 11:34:47.261291 25272 raft_consensus.cc:2258] T 16cc68ea874b48f19573f4e739edcc74 P 1103ed0eb8974fd8bc3eb2955eacbc78 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:34:47.266816 25245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:34:47.236603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54644 (local address 127.8.18.193:43511)
0810 11:34:47.236804 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0810 11:34:47.236813 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 11:34:47.236969 (+ 156us) server_negotiation.cc:407] Connection header received
0810 11:34:47.237242 (+ 273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:34:47.237248 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:34:47.237350 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:34:47.237504 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:34:47.240158 (+ 2654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:34:47.241502 (+ 1344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:34:47.244331 (+ 2829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:34:47.245095 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:34:47.245252 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:34:47.255353 (+ 10101us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:34:47.255372 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:34:47.255397 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:34:47.255452 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:34:47.260300 (+ 4848us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:34:47.262729 (+ 2429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:34:47.262745 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:34:47.262751 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:34:47.263075 (+ 324us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:34:47.263897 (+ 822us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:34:47.263905 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:34:47.263911 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:34:47.264593 (+ 682us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:34:47.264995 (+ 402us) server_negotiation.cc:1035] Waiting for connection context
0810 11:34:47.265942 (+ 947us) server_negotiation.cc:299] Negotiation successful
0810 11:34:47.266391 (+ 449us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260810 11:34:47.266840 25273 tablet_replica.cc:330] T 16cc68ea874b48f19573f4e739edcc74 P f87e487a91b542408b4a24ffc52421be: stopping tablet replica
I20260810 11:34:47.267349 25273 raft_consensus.cc:2229] T 16cc68ea874b48f19573f4e739edcc74 P f87e487a91b542408b4a24ffc52421be [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 11:34:47.267984 25273 raft_consensus.cc:2258] T 16cc68ea874b48f19573f4e739edcc74 P f87e487a91b542408b4a24ffc52421be [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:34:47.268925 24957 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:34:47.270838 25272 ts_tablet_manager.cc:1905] T 16cc68ea874b48f19573f4e739edcc74 P 1103ed0eb8974fd8bc3eb2955eacbc78: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:34:47.281265 25272 ts_tablet_manager.cc:1918] T 16cc68ea874b48f19573f4e739edcc74 P 1103ed0eb8974fd8bc3eb2955eacbc78: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 11:34:47.281558 25272 log.cc:1196] T 16cc68ea874b48f19573f4e739edcc74 P 1103ed0eb8974fd8bc3eb2955eacbc78: Deleting WAL directory at /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/ts-1/wal/wals/16cc68ea874b48f19573f4e739edcc74
I20260810 11:34:47.282280 25272 ts_tablet_manager.cc:1939] T 16cc68ea874b48f19573f4e739edcc74 P 1103ed0eb8974fd8bc3eb2955eacbc78: Deleting consensus metadata
I20260810 11:34:47.284927 24760 catalog_manager.cc:4841] TS 1103ed0eb8974fd8bc3eb2955eacbc78 (127.8.18.194:34511): tablet 16cc68ea874b48f19573f4e739edcc74 (table test-table [id=f905ef31a5434d799b04b887518d4e29]) successfully deleted
I20260810 11:34:47.287300 25273 ts_tablet_manager.cc:1905] T 16cc68ea874b48f19573f4e739edcc74 P f87e487a91b542408b4a24ffc52421be: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:34:47.296536 25273 ts_tablet_manager.cc:1918] T 16cc68ea874b48f19573f4e739edcc74 P f87e487a91b542408b4a24ffc52421be: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 11:34:47.296774 25273 log.cc:1196] T 16cc68ea874b48f19573f4e739edcc74 P f87e487a91b542408b4a24ffc52421be: Deleting WAL directory at /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/ts-2/wal/wals/16cc68ea874b48f19573f4e739edcc74
I20260810 11:34:47.297394 25273 ts_tablet_manager.cc:1939] T 16cc68ea874b48f19573f4e739edcc74 P f87e487a91b542408b4a24ffc52421be: Deleting consensus metadata
I20260810 11:34:47.299604 24762 catalog_manager.cc:4841] TS f87e487a91b542408b4a24ffc52421be (127.8.18.195:40573): tablet 16cc68ea874b48f19573f4e739edcc74 (table test-table [id=f905ef31a5434d799b04b887518d4e29]) successfully deleted
I20260810 11:34:47.385618 25271 ts_tablet_manager.cc:1918] T 16cc68ea874b48f19573f4e739edcc74 P 8a82dd1f6f654360b68d7ff014fe5e71: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 11:34:47.386039 25271 log.cc:1196] T 16cc68ea874b48f19573f4e739edcc74 P 8a82dd1f6f654360b68d7ff014fe5e71: Deleting WAL directory at /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/ts-0/wal/wals/16cc68ea874b48f19573f4e739edcc74
I20260810 11:34:47.386858 25271 ts_tablet_manager.cc:1939] T 16cc68ea874b48f19573f4e739edcc74 P 8a82dd1f6f654360b68d7ff014fe5e71: Deleting consensus metadata
I20260810 11:34:47.389691 24761 catalog_manager.cc:4841] TS 8a82dd1f6f654360b68d7ff014fe5e71 (127.8.18.193:43511): tablet 16cc68ea874b48f19573f4e739edcc74 (table test-table [id=f905ef31a5434d799b04b887518d4e29]) successfully deleted
I20260810 11:34:47.389880 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 24813
I20260810 11:34:47.413121 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 24966
I20260810 11:34:47.431480 25241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:34:47.414072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51484 (local address 127.8.18.194:34511)
0810 11:34:47.414284 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0810 11:34:47.415607 (+ 1323us) server_negotiation.cc:399] Waiting for connection header
0810 11:34:47.415634 (+ 27us) server_negotiation.cc:407] Connection header received
0810 11:34:47.415713 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:34:47.415718 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:34:47.415819 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:34:47.415948 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:34:47.417399 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:34:47.418221 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:34:47.419673 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:34:47.420121 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:34:47.420276 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:34:47.423978 (+ 3702us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:34:47.423997 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:34:47.424025 (+ 28us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:34:47.424078 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:34:47.427949 (+ 3871us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:34:47.428884 (+ 935us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:34:47.428892 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:34:47.428895 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:34:47.429036 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:34:47.429509 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:34:47.429515 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:34:47.429520 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:34:47.430000 (+ 480us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:34:47.430349 (+ 349us) server_negotiation.cc:1035] Waiting for connection context
0810 11:34:47.430733 (+ 384us) server_negotiation.cc:299] Negotiation successful
0810 11:34:47.430982 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260810 11:34:47.432216 25090 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:34:47.527551 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 24966
I20260810 11:34:47.549144 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 25100
I20260810 11:34:47.564343 25244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:34:47.550073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36486 (local address 127.8.18.195:40573)
0810 11:34:47.550409 (+ 336us) server_negotiation.cc:206] Beginning negotiation
0810 11:34:47.550416 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:34:47.550440 (+ 24us) server_negotiation.cc:407] Connection header received
0810 11:34:47.550556 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:34:47.550565 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:34:47.550673 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:34:47.550805 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:34:47.552244 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:34:47.553021 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:34:47.554560 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:34:47.554984 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:34:47.555098 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:34:47.558094 (+ 2996us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:34:47.558113 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:34:47.558131 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:34:47.558174 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:34:47.561043 (+ 2869us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:34:47.561952 (+ 909us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:34:47.561960 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:34:47.561963 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:34:47.562126 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:34:47.562513 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:34:47.562517 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:34:47.562519 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:34:47.562956 (+ 437us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:34:47.563304 (+ 348us) server_negotiation.cc:1035] Waiting for connection context
0810 11:34:47.563686 (+ 382us) server_negotiation.cc:299] Negotiation successful
0810 11:34:47.563972 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":196}
I20260810 11:34:47.565097 25225 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:34:47.663481 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 25100
I20260810 11:34:47.683868 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 24742
I20260810 11:34:47.685257 24806 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:34:47.808022 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 24742
I20260810 11:35:07.834904 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.18.254:42587
--webserver_interface=127.8.18.254
--webserver_port=39739
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.254.keytab
--principal=kudu/127.8.18.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.18.212:33987
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.18.254:42587
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:35:08.216512 25278 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:35:08.216835 25278 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:35:08.217005 25278 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:35:08.224820 25278 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260810 11:35:08.224936 25278 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260810 11:35:08.224999 25278 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260810 11:35:08.225080 25278 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 11:35:08.225157 25278 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:35:08.225266 25278 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:35:08.225342 25278 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 11:35:08.225445 25278 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 11:35:08.236362 25278 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.18.212:33987
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.8.18.254:42587
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.254.keytab
--principal=kudu/127.8.18.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.18.254:42587
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.18.254
--webserver_port=39739
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:35:08.237844 25278 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:35:08.239291 25278 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:35:08.252718 25286 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:35:08.252774 25284 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:35:08.252800 25283 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:35:08.254158 25278 server_base.cc:1034] running on GCE node
I20260810 11:35:08.255086 25278 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:35:08.256668 25278 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:35:08.257959 25278 hybrid_clock.cc:648] HybridClock initialized: now 1786361708257886 us; error 54 us; skew 500 ppm
I20260810 11:35:08.258378 25278 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:35:08 dist-test-slave-srgv krb5kdc[24177](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361708, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:35:08.262002 25278 init.cc:377] Logged in from keytab as kudu/127.8.18.254@KRBTEST.COM (short username kudu)
I20260810 11:35:08.263976 25278 webserver.cc:460] Webserver started at http://127.8.18.254:39739/ using document root <none> and password file <none>
I20260810 11:35:08.264797 25278 fs_manager.cc:359] Metadata directory not provided
I20260810 11:35:08.264955 25278 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:35:08.271229 25278 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260810 11:35:08.274624 25293 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:35:08.276130 25278 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260810 11:35:08.276391 25278 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/master-0/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/master-0/wal
uuid: "25ef7a60224c4f7bb5147d7c5ed8251c"
format_stamp: "Formatted at 2026-08-10 11:34:21 on dist-test-slave-srgv"
server_key: "59e635f02b278030db5011de502297ee"
server_key_iv: "b7d235b3fe008091868335d912f66813"
server_key_version: "encryptionkey@0"
I20260810 11:35:08.277513 25278 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:35:08.300886 25278 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:35:08.306756 25278 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:35:08.307034 25278 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:35:08.328330 25278 rpc_server.cc:307] RPC server started. Bound to: 127.8.18.254:42587
I20260810 11:35:08.328341 25345 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.18.254:42587 every 8 connection(s)
I20260810 11:35:08.330142 25278 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
I20260810 11:35:08.337566 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 25278
I20260810 11:35:08.338325 25346 sys_catalog.cc:263] Verifying existing consensus state
I20260810 11:35:08.338577 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.18.193:43511
--local_ip_for_outbound_sockets=127.8.18.193
--tserver_master_addrs=127.8.18.254:42587
--webserver_port=38413
--webserver_interface=127.8.18.193
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.193.keytab
--principal=kudu/127.8.18.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.18.212:33987
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 11:35:08.342231 25346 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 25ef7a60224c4f7bb5147d7c5ed8251c: Bootstrap starting.
I20260810 11:35:08.365188 25346 log.cc:824] T 00000000000000000000000000000000 P 25ef7a60224c4f7bb5147d7c5ed8251c: Log is configured to *not* fsync() on all Append() calls
I20260810 11:35:08.383533 25346 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 25ef7a60224c4f7bb5147d7c5ed8251c: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 11:35:08.384210 25346 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 25ef7a60224c4f7bb5147d7c5ed8251c: Bootstrap complete.
I20260810 11:35:08.394292 25346 raft_consensus.cc:348] T 00000000000000000000000000000000 P 25ef7a60224c4f7bb5147d7c5ed8251c [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25ef7a60224c4f7bb5147d7c5ed8251c" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 42587 } }
I20260810 11:35:08.395361 25346 raft_consensus.cc:729] T 00000000000000000000000000000000 P 25ef7a60224c4f7bb5147d7c5ed8251c [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25ef7a60224c4f7bb5147d7c5ed8251c, State: Initialized, Role: FOLLOWER
I20260810 11:35:08.396166 25346 consensus_queue.cc:260] T 00000000000000000000000000000000 P 25ef7a60224c4f7bb5147d7c5ed8251c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25ef7a60224c4f7bb5147d7c5ed8251c" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 42587 } }
I20260810 11:35:08.396462 25346 raft_consensus.cc:388] T 00000000000000000000000000000000 P 25ef7a60224c4f7bb5147d7c5ed8251c [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 11:35:08.396613 25346 raft_consensus.cc:482] T 00000000000000000000000000000000 P 25ef7a60224c4f7bb5147d7c5ed8251c [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 11:35:08.396818 25346 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 25ef7a60224c4f7bb5147d7c5ed8251c [term 2 FOLLOWER]: Advancing to term 3
I20260810 11:35:08.401492 25346 raft_consensus.cc:504] T 00000000000000000000000000000000 P 25ef7a60224c4f7bb5147d7c5ed8251c [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25ef7a60224c4f7bb5147d7c5ed8251c" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 42587 } }
I20260810 11:35:08.402083 25346 leader_election.cc:302] T 00000000000000000000000000000000 P 25ef7a60224c4f7bb5147d7c5ed8251c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 25ef7a60224c4f7bb5147d7c5ed8251c; no voters:
I20260810 11:35:08.402848 25346 leader_election.cc:288] T 00000000000000000000000000000000 P 25ef7a60224c4f7bb5147d7c5ed8251c [CANDIDATE]: Term 3 election: Requested vote from peers
I20260810 11:35:08.403000 25350 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 25ef7a60224c4f7bb5147d7c5ed8251c [term 3 FOLLOWER]: Leader election won for term 3
I20260810 11:35:08.404448 25350 raft_consensus.cc:686] T 00000000000000000000000000000000 P 25ef7a60224c4f7bb5147d7c5ed8251c [term 3 LEADER]: Becoming Leader. State: Replica: 25ef7a60224c4f7bb5147d7c5ed8251c, State: Running, Role: LEADER
I20260810 11:35:08.405077 25350 consensus_queue.cc:237] T 00000000000000000000000000000000 P 25ef7a60224c4f7bb5147d7c5ed8251c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25ef7a60224c4f7bb5147d7c5ed8251c" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 42587 } }
I20260810 11:35:08.405642 25346 sys_catalog.cc:564] T 00000000000000000000000000000000 P 25ef7a60224c4f7bb5147d7c5ed8251c [sys.catalog]: configured and running, proceeding with master startup.
I20260810 11:35:08.407817 25352 sys_catalog.cc:455] T 00000000000000000000000000000000 P 25ef7a60224c4f7bb5147d7c5ed8251c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 25ef7a60224c4f7bb5147d7c5ed8251c. Latest consensus state: current_term: 3 leader_uuid: "25ef7a60224c4f7bb5147d7c5ed8251c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25ef7a60224c4f7bb5147d7c5ed8251c" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 42587 } } }
I20260810 11:35:08.408252 25352 sys_catalog.cc:458] T 00000000000000000000000000000000 P 25ef7a60224c4f7bb5147d7c5ed8251c [sys.catalog]: This master's current role is: LEADER
I20260810 11:35:08.408874 25351 sys_catalog.cc:455] T 00000000000000000000000000000000 P 25ef7a60224c4f7bb5147d7c5ed8251c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "25ef7a60224c4f7bb5147d7c5ed8251c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25ef7a60224c4f7bb5147d7c5ed8251c" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 42587 } } }
I20260810 11:35:08.409219 25351 sys_catalog.cc:458] T 00000000000000000000000000000000 P 25ef7a60224c4f7bb5147d7c5ed8251c [sys.catalog]: This master's current role is: LEADER
I20260810 11:35:08.414258 25356 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 11:35:08.424372 25356 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 11:35:08.426424 25356 catalog_manager.cc:1259] Loaded cluster ID: e2a8e20598364125b81bb3c2699a96d8
I20260810 11:35:08.426561 25356 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 11:35:08.430819 25356 catalog_manager.cc:1504] Loading token signing keys...
I20260810 11:35:08.432964 25356 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 25ef7a60224c4f7bb5147d7c5ed8251c: Loaded TSK: 0
I20260810 11:35:08.433138 25356 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 25ef7a60224c4f7bb5147d7c5ed8251c: Loaded TSK: 1
F20260810 11:35:08.434365 25356 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7faecc9e29ad google::LogMessage::Fail() at ??:0
@ 0x7faecc9e588a google::LogMessage::SendToLog() at ??:0
@ 0x7faecc9e24de google::LogMessage::Flush() at ??:0
@ 0x7faecc9e3a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7faed07467d2 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7faed073573e kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7faee24111c0 kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7faee2410783 kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7faee249c2a7 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_10::operator()() at ??:0
@ 0x7faee249c0c4 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x7faee24e65e9 std::function<>::operator()() at ??:0
@ 0x7faee2414cdd kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7faee2412c36 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7faee249a610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7faee249a439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7faecf62e345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7faecf63191c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7faecf631799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7faecf60ceaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7faed28e66db start_thread at ??:0
@ 0x7faecb87c61f clone at ??:0
W20260810 11:35:08.757143 25348 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:35:08.757575 25348 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:35:08.757691 25348 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:35:08.765525 25348 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:35:08.765666 25348 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:35:08.765787 25348 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.18.193
I20260810 11:35:08.776273 25348 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.18.212:33987
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.193.keytab
--principal=kudu/127.8.18.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.18.193:43511
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.18.193
--webserver_port=38413
--tserver_master_addrs=127.8.18.254:42587
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.18.193
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:35:08.777695 25348 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:35:08.779035 25348 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:35:08.793948 25397 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:35:08.794026 25395 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:35:08.793977 25394 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:35:08.794874 25348 server_base.cc:1034] running on GCE node
I20260810 11:35:08.795614 25348 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:35:08.796661 25348 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:35:08.797897 25348 hybrid_clock.cc:648] HybridClock initialized: now 1786361708797852 us; error 33 us; skew 500 ppm
I20260810 11:35:08.798238 25348 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:35:08 dist-test-slave-srgv krb5kdc[24177](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361708, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:35:08.802050 25348 init.cc:377] Logged in from keytab as kudu/127.8.18.193@KRBTEST.COM (short username kudu)
I20260810 11:35:08.803776 25348 webserver.cc:460] Webserver started at http://127.8.18.193:38413/ using document root <none> and password file <none>
I20260810 11:35:08.804608 25348 fs_manager.cc:359] Metadata directory not provided
I20260810 11:35:08.804762 25348 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:35:08.810907 25348 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260810 11:35:08.814364 25404 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:35:08.815966 25348 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260810 11:35:08.816237 25348 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/ts-0/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/ts-0/wal
uuid: "8a82dd1f6f654360b68d7ff014fe5e71"
format_stamp: "Formatted at 2026-08-10 11:34:22 on dist-test-slave-srgv"
server_key: "d17b1c80112eb8b90b6e989b0c672614"
server_key_iv: "bb6e0401ef0735452bbcac4e2c4c779f"
server_key_version: "encryptionkey@0"
I20260810 11:35:08.817395 25348 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:35:08.832121 25348 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:35:08.837054 25348 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:35:08.837463 25348 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:35:08.838627 25348 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:35:08.840258 25348 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:35:08.840415 25348 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:35:08.840560 25348 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:35:08.840655 25348 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:35:08.878762 25348 rpc_server.cc:307] RPC server started. Bound to: 127.8.18.193:43511
I20260810 11:35:08.878868 25517 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.18.193:43511 every 8 connection(s)
I20260810 11:35:08.880484 25348 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/ts-0/data/info.pb
I20260810 11:35:08.881146 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 25348
I20260810 11:35:08.882113 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.18.194:34511
--local_ip_for_outbound_sockets=127.8.18.194
--tserver_master_addrs=127.8.18.254:42587
--webserver_port=44741
--webserver_interface=127.8.18.194
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.194.keytab
--principal=kudu/127.8.18.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.18.212:33987
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 11:35:08.886651 25520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:08.884280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:42587 (local address 127.8.18.193:43785)
0810 11:35:08.885265 (+ 985us) negotiation.cc:107] Waiting for socket to connect
0810 11:35:08.885587 (+ 322us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.254:42587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":655,"thread_start_us":437,"threads_started":1}
W20260810 11:35:08.887539 25518 heartbeater.cc:644] Failed to heartbeat to 127.8.18.254:42587 (0 consecutive failures): Network error: Failed to ping master at 127.8.18.254:42587: Client connection negotiation failed: client connection to 127.8.18.254:42587: connect: Connection refused (error 111)
I20260810 11:35:08.888554 25520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:08.888165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:42587 (local address 127.8.18.193:52821)
0810 11:35:08.888344 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0810 11:35:08.888431 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.254:42587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260810 11:35:08.889724 25520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:08.889283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:42587 (local address 127.8.18.193:56243)
0810 11:35:08.889488 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0810 11:35:08.889589 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.254:42587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260810 11:35:08.890017 25518 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260810 11:35:09.257334 25521 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:35:09.257787 25521 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:35:09.257900 25521 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:35:09.265885 25521 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:35:09.266019 25521 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:35:09.266141 25521 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.18.194
I20260810 11:35:09.276551 25521 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.18.212:33987
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.194.keytab
--principal=kudu/127.8.18.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.18.194:34511
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.18.194
--webserver_port=44741
--tserver_master_addrs=127.8.18.254:42587
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.18.194
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:35:09.277989 25521 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:35:09.279323 25521 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:35:09.294387 25527 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:35:09.294519 25526 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:35:09.294557 25529 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:35:09.295890 25521 server_base.cc:1034] running on GCE node
I20260810 11:35:09.296649 25521 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:35:09.297863 25521 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:35:09.299114 25521 hybrid_clock.cc:648] HybridClock initialized: now 1786361709299061 us; error 46 us; skew 500 ppm
I20260810 11:35:09.299472 25521 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:35:09 dist-test-slave-srgv krb5kdc[24177](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361709, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:35:09.303356 25521 init.cc:377] Logged in from keytab as kudu/127.8.18.194@KRBTEST.COM (short username kudu)
I20260810 11:35:09.305457 25521 webserver.cc:460] Webserver started at http://127.8.18.194:44741/ using document root <none> and password file <none>
I20260810 11:35:09.306484 25521 fs_manager.cc:359] Metadata directory not provided
I20260810 11:35:09.306674 25521 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:35:09.313359 25521 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.003s
I20260810 11:35:09.316455 25536 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:35:09.318156 25521 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 11:35:09.318382 25521 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/ts-1/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/ts-1/wal
uuid: "1103ed0eb8974fd8bc3eb2955eacbc78"
format_stamp: "Formatted at 2026-08-10 11:34:23 on dist-test-slave-srgv"
server_key: "bddc7322a41df9efaed932734d963059"
server_key_iv: "1fa90db762df1c07e2bda8f2688f461b"
server_key_version: "encryptionkey@0"
I20260810 11:35:09.319504 25521 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:35:09.336841 25521 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:35:09.342120 25521 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:35:09.342469 25521 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:35:09.343670 25521 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:35:09.345283 25521 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:35:09.345480 25521 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:35:09.345633 25521 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:35:09.345747 25521 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:35:09.382990 25521 rpc_server.cc:307] RPC server started. Bound to: 127.8.18.194:34511
I20260810 11:35:09.383054 25650 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.18.194:34511 every 8 connection(s)
I20260810 11:35:09.384647 25521 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/ts-1/data/info.pb
I20260810 11:35:09.386564 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 25521
I20260810 11:35:09.387317 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.18.195:40573
--local_ip_for_outbound_sockets=127.8.18.195
--tserver_master_addrs=127.8.18.254:42587
--webserver_port=46223
--webserver_interface=127.8.18.195
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.195.keytab
--principal=kudu/127.8.18.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.18.212:33987
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 11:35:09.399153 25653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:09.396304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:42587 (local address 127.8.18.194:58107)
0810 11:35:09.397345 (+ 1041us) negotiation.cc:107] Waiting for socket to connect
0810 11:35:09.397759 (+ 414us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.254:42587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":703,"thread_start_us":451,"threads_started":1}
W20260810 11:35:09.399997 25651 heartbeater.cc:644] Failed to heartbeat to 127.8.18.254:42587 (0 consecutive failures): Network error: Failed to ping master at 127.8.18.254:42587: Client connection negotiation failed: client connection to 127.8.18.254:42587: connect: Connection refused (error 111)
I20260810 11:35:09.401185 25653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:09.400726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:42587 (local address 127.8.18.194:47429)
0810 11:35:09.400951 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0810 11:35:09.401051 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.254:42587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260810 11:35:09.402187 25653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:09.401815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:42587 (local address 127.8.18.194:39737)
0810 11:35:09.401973 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0810 11:35:09.402071 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.254:42587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260810 11:35:09.402458 25651 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260810 11:35:09.760676 25654 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:35:09.761056 25654 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:35:09.761229 25654 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:35:09.768821 25654 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:35:09.769016 25654 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:35:09.769229 25654 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.18.195
I20260810 11:35:09.779477 25654 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.18.212:33987
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.195.keytab
--principal=kudu/127.8.18.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.18.195:40573
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.18.195
--webserver_port=46223
--tserver_master_addrs=127.8.18.254:42587
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.18.195
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:35:09.780933 25654 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:35:09.782377 25654 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:35:09.796723 25660 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:35:09.796633 25662 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:35:09.796756 25659 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:35:09.797887 25654 server_base.cc:1034] running on GCE node
I20260810 11:35:09.798619 25654 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:35:09.799700 25654 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:35:09.800978 25654 hybrid_clock.cc:648] HybridClock initialized: now 1786361709800914 us; error 56 us; skew 500 ppm
I20260810 11:35:09.801393 25654 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:35:09 dist-test-slave-srgv krb5kdc[24177](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361709, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:35:09.805254 25654 init.cc:377] Logged in from keytab as kudu/127.8.18.195@KRBTEST.COM (short username kudu)
I20260810 11:35:09.807039 25654 webserver.cc:460] Webserver started at http://127.8.18.195:46223/ using document root <none> and password file <none>
I20260810 11:35:09.807885 25654 fs_manager.cc:359] Metadata directory not provided
I20260810 11:35:09.808068 25654 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:35:09.814844 25654 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260810 11:35:09.818411 25670 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:35:09.820129 25654 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260810 11:35:09.820389 25654 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/ts-2/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/ts-2/wal
uuid: "f87e487a91b542408b4a24ffc52421be"
format_stamp: "Formatted at 2026-08-10 11:34:23 on dist-test-slave-srgv"
server_key: "f937034459273ddb79f12ef371db5203"
server_key_iv: "c5654e1e7bfbd521d2210a671cc5a52a"
server_key_version: "encryptionkey@0"
I20260810 11:35:09.821681 25654 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:35:09.853176 25654 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:35:09.857901 25654 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:35:09.858250 25654 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:35:09.859375 25654 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:35:09.860976 25654 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:35:09.861119 25654 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:35:09.861274 25654 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:35:09.861370 25654 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:35:09.891768 25762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:09.890653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:42587 (local address 127.8.18.193:40093)
0810 11:35:09.891269 (+ 616us) negotiation.cc:107] Waiting for socket to connect
0810 11:35:09.891440 (+ 171us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.254:42587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":451,"thread_start_us":285,"threads_started":1}
I20260810 11:35:09.899493 25654 rpc_server.cc:307] RPC server started. Bound to: 127.8.18.195:40573
I20260810 11:35:09.899588 25789 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.18.195:40573 every 8 connection(s)
I20260810 11:35:09.901203 25654 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786361401922362-8267-0/minicluster-data/ts-2/data/info.pb
I20260810 11:35:09.901827 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 25654
I20260810 11:35:09.907516 25793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:09.905321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:42587 (local address 127.8.18.195:50239)
0810 11:35:09.906223 (+ 902us) negotiation.cc:107] Waiting for socket to connect
0810 11:35:09.906483 (+ 260us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.254:42587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":634,"thread_start_us":328,"threads_started":1}
W20260810 11:35:09.908375 25790 heartbeater.cc:644] Failed to heartbeat to 127.8.18.254:42587 (0 consecutive failures): Network error: Failed to ping master at 127.8.18.254:42587: Client connection negotiation failed: client connection to 127.8.18.254:42587: connect: Connection refused (error 111)
I20260810 11:35:09.909464 25793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:09.909042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:42587 (local address 127.8.18.195:44291)
0810 11:35:09.909211 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0810 11:35:09.909291 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.254:42587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260810 11:35:09.910423 25793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:09.910084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:42587 (local address 127.8.18.195:56447)
0810 11:35:09.910219 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0810 11:35:09.910306 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.254:42587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260810 11:35:09.910689 25790 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260810 11:35:10.404353 25794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:10.403230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:42587 (local address 127.8.18.194:36787)
0810 11:35:10.403859 (+ 629us) negotiation.cc:107] Waiting for socket to connect
0810 11:35:10.404000 (+ 141us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.254:42587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":439,"thread_start_us":270,"threads_started":1}
I20260810 11:35:10.893734 25795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:10.892811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:42587 (local address 127.8.18.193:37305)
0810 11:35:10.893344 (+ 533us) negotiation.cc:107] Waiting for socket to connect
0810 11:35:10.893541 (+ 197us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.254:42587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":422,"thread_start_us":276,"threads_started":1}
I20260810 11:35:10.912345 25796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:10.911389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:42587 (local address 127.8.18.195:43165)
0810 11:35:10.911970 (+ 581us) negotiation.cc:107] Waiting for socket to connect
0810 11:35:10.912086 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.254:42587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":458,"thread_start_us":305,"threads_started":1}
I20260810 11:35:11.406549 25797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:11.405689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:42587 (local address 127.8.18.194:38655)
0810 11:35:11.406220 (+ 531us) negotiation.cc:107] Waiting for socket to connect
0810 11:35:11.406354 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.254:42587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":412,"thread_start_us":259,"threads_started":1}
I20260810 11:35:11.895701 25798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:11.894790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:42587 (local address 127.8.18.193:38769)
0810 11:35:11.895408 (+ 618us) negotiation.cc:107] Waiting for socket to connect
0810 11:35:11.895508 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.254:42587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":510,"thread_start_us":287,"threads_started":1}
I20260810 11:35:11.914500 25799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:11.913601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:42587 (local address 127.8.18.195:50461)
0810 11:35:11.914204 (+ 603us) negotiation.cc:107] Waiting for socket to connect
0810 11:35:11.914317 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.254:42587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":488,"thread_start_us":274,"threads_started":1}
I20260810 11:35:12.408818 25800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:12.407922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:42587 (local address 127.8.18.194:57127)
0810 11:35:12.408524 (+ 602us) negotiation.cc:107] Waiting for socket to connect
0810 11:35:12.408622 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.254:42587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":414,"thread_start_us":286,"threads_started":1}
I20260810 11:35:12.897753 25801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:12.896787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:42587 (local address 127.8.18.193:38551)
0810 11:35:12.897369 (+ 582us) negotiation.cc:107] Waiting for socket to connect
0810 11:35:12.897547 (+ 178us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.254:42587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":463,"thread_start_us":296,"threads_started":1}
I20260810 11:35:12.916736 25802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:12.915718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:42587 (local address 127.8.18.195:51827)
0810 11:35:12.916406 (+ 688us) negotiation.cc:107] Waiting for socket to connect
0810 11:35:12.916533 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.254:42587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":547,"thread_start_us":388,"threads_started":1}
I20260810 11:35:13.410820 25803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:13.409893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:42587 (local address 127.8.18.194:56829)
0810 11:35:13.410531 (+ 638us) negotiation.cc:107] Waiting for socket to connect
0810 11:35:13.410621 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.254:42587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":505,"thread_start_us":338,"threads_started":1}
I20260810 11:35:13.899936 25804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:13.898957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:42587 (local address 127.8.18.193:57257)
0810 11:35:13.899631 (+ 674us) negotiation.cc:107] Waiting for socket to connect
0810 11:35:13.899745 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.254:42587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":535,"thread_start_us":416,"threads_started":1}
I20260810 11:35:13.918795 25805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:13.917987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:42587 (local address 127.8.18.195:41313)
0810 11:35:13.918518 (+ 531us) negotiation.cc:107] Waiting for socket to connect
0810 11:35:13.918606 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.254:42587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":417,"thread_start_us":290,"threads_started":1}
I20260810 11:35:14.412907 25806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:14.411993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:42587 (local address 127.8.18.194:52151)
0810 11:35:14.412625 (+ 632us) negotiation.cc:107] Waiting for socket to connect
0810 11:35:14.412723 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.254:42587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":487,"thread_start_us":319,"threads_started":1}
I20260810 11:35:14.902140 25807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:14.901181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:42587 (local address 127.8.18.193:35547)
0810 11:35:14.901839 (+ 658us) negotiation.cc:107] Waiting for socket to connect
0810 11:35:14.901945 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.254:42587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":545,"thread_start_us":387,"threads_started":1}
I20260810 11:35:14.920899 25808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:14.919970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:42587 (local address 127.8.18.195:37071)
0810 11:35:14.920619 (+ 649us) negotiation.cc:107] Waiting for socket to connect
0810 11:35:14.920711 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.254:42587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":529,"thread_start_us":321,"threads_started":1}
I20260810 11:35:15.415036 25809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:15.414167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:42587 (local address 127.8.18.194:36169)
0810 11:35:15.414747 (+ 580us) negotiation.cc:107] Waiting for socket to connect
0810 11:35:15.414857 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.254:42587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":305,"threads_started":1}
I20260810 11:35:15.904070 25810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:15.903215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:42587 (local address 127.8.18.193:40299)
0810 11:35:15.903751 (+ 536us) negotiation.cc:107] Waiting for socket to connect
0810 11:35:15.903837 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.254:42587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":427,"thread_start_us":233,"threads_started":1}
I20260810 11:35:15.922979 25811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:15.922067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:42587 (local address 127.8.18.195:47693)
0810 11:35:15.922686 (+ 619us) negotiation.cc:107] Waiting for socket to connect
0810 11:35:15.922802 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.254:42587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":490,"thread_start_us":308,"threads_started":1}
I20260810 11:35:16.417109 25813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:16.416173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:42587 (local address 127.8.18.194:46093)
0810 11:35:16.416779 (+ 606us) negotiation.cc:107] Waiting for socket to connect
0810 11:35:16.416878 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.254:42587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":469,"thread_start_us":256,"threads_started":1}
I20260810 11:35:16.906252 25814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:16.905303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:42587 (local address 127.8.18.193:55425)
0810 11:35:16.905964 (+ 661us) negotiation.cc:107] Waiting for socket to connect
0810 11:35:16.906071 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.254:42587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":545,"thread_start_us":381,"threads_started":1}
I20260810 11:35:16.925227 25815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:16.924243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:42587 (local address 127.8.18.195:34025)
0810 11:35:16.924839 (+ 596us) negotiation.cc:107] Waiting for socket to connect
0810 11:35:16.925007 (+ 168us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.254:42587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":468,"thread_start_us":291,"threads_started":1}
I20260810 11:35:17.418947 25816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:17.418167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:42587 (local address 127.8.18.194:40539)
0810 11:35:17.418684 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0810 11:35:17.418771 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.254:42587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":413,"thread_start_us":283,"threads_started":1}
I20260810 11:35:17.908378 25817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:17.907488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:42587 (local address 127.8.18.193:57307)
0810 11:35:17.908092 (+ 604us) negotiation.cc:107] Waiting for socket to connect
0810 11:35:17.908191 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.254:42587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":444,"thread_start_us":271,"threads_started":1}
I20260810 11:35:17.927273 25819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:17.926382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:42587 (local address 127.8.18.195:44409)
0810 11:35:17.926961 (+ 579us) negotiation.cc:107] Waiting for socket to connect
0810 11:35:17.927052 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.254:42587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":470,"thread_start_us":315,"threads_started":1}
I20260810 11:35:18.420969 25820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:18.419951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:42587 (local address 127.8.18.194:56655)
0810 11:35:18.420537 (+ 586us) negotiation.cc:107] Waiting for socket to connect
0810 11:35:18.420696 (+ 159us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.254:42587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":436,"thread_start_us":268,"threads_started":1}
I20260810 11:35:18.910516 25821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:18.909662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:42587 (local address 127.8.18.193:56351)
0810 11:35:18.910244 (+ 582us) negotiation.cc:107] Waiting for socket to connect
0810 11:35:18.910340 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.254:42587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":431,"thread_start_us":290,"threads_started":1}
I20260810 11:35:18.929288 25822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:18.928323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:42587 (local address 127.8.18.195:44619)
0810 11:35:18.928887 (+ 564us) negotiation.cc:107] Waiting for socket to connect
0810 11:35:18.929099 (+ 212us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.254:42587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":445,"thread_start_us":284,"threads_started":1}
I20260810 11:35:19.423118 25823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:19.422200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:42587 (local address 127.8.18.194:35255)
0810 11:35:19.422727 (+ 527us) negotiation.cc:107] Waiting for socket to connect
0810 11:35:19.422919 (+ 192us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.254:42587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":408,"thread_start_us":276,"threads_started":1}
I20260810 11:35:19.912719 25824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:19.911753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:42587 (local address 127.8.18.193:54937)
0810 11:35:19.912410 (+ 657us) negotiation.cc:107] Waiting for socket to connect
0810 11:35:19.912494 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.254:42587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":545,"thread_start_us":418,"threads_started":1}
I20260810 11:35:19.931196 25825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:19.930387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:42587 (local address 127.8.18.195:45553)
0810 11:35:19.930896 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0810 11:35:19.930986 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.254:42587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":398,"thread_start_us":278,"threads_started":1}
I20260810 11:35:20.425177 25826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:20.424258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:42587 (local address 127.8.18.194:35731)
0810 11:35:20.424829 (+ 571us) negotiation.cc:107] Waiting for socket to connect
0810 11:35:20.424936 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.254:42587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":416,"thread_start_us":253,"threads_started":1}
I20260810 11:35:20.914721 25827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:20.913876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:42587 (local address 127.8.18.193:41353)
0810 11:35:20.914434 (+ 558us) negotiation.cc:107] Waiting for socket to connect
0810 11:35:20.914548 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.254:42587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":439,"thread_start_us":320,"threads_started":1}
I20260810 11:35:20.933072 25828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:20.932219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:42587 (local address 127.8.18.195:53905)
0810 11:35:20.932774 (+ 555us) negotiation.cc:107] Waiting for socket to connect
0810 11:35:20.932876 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.254:42587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":443,"thread_start_us":271,"threads_started":1}
I20260810 11:35:21.427258 25829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:21.426418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:42587 (local address 127.8.18.194:50919)
0810 11:35:21.426991 (+ 573us) negotiation.cc:107] Waiting for socket to connect
0810 11:35:21.427084 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.254:42587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":438,"thread_start_us":335,"threads_started":1}
I20260810 11:35:21.916954 25830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:21.916046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:42587 (local address 127.8.18.193:41589)
0810 11:35:21.916642 (+ 596us) negotiation.cc:107] Waiting for socket to connect
0810 11:35:21.916753 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.254:42587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":468,"thread_start_us":287,"threads_started":1}
I20260810 11:35:21.935220 25831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:21.934369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:42587 (local address 127.8.18.195:52795)
0810 11:35:21.934925 (+ 556us) negotiation.cc:107] Waiting for socket to connect
0810 11:35:21.935029 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.254:42587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":417,"thread_start_us":283,"threads_started":1}
I20260810 11:35:22.429167 25832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:22.428283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:42587 (local address 127.8.18.194:52065)
0810 11:35:22.428808 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0810 11:35:22.428916 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.254:42587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":417,"thread_start_us":266,"threads_started":1}
I20260810 11:35:22.919065 25833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:22.918221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:42587 (local address 127.8.18.193:60047)
0810 11:35:22.918775 (+ 554us) negotiation.cc:107] Waiting for socket to connect
0810 11:35:22.918870 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.254:42587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":412,"thread_start_us":257,"threads_started":1}
I20260810 11:35:22.937252 25834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:22.936330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:42587 (local address 127.8.18.195:41841)
0810 11:35:22.936895 (+ 565us) negotiation.cc:107] Waiting for socket to connect
0810 11:35:22.937007 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.254:42587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":449,"thread_start_us":293,"threads_started":1}
I20260810 11:35:23.431362 25835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:23.430387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:42587 (local address 127.8.18.194:59583)
0810 11:35:23.430995 (+ 608us) negotiation.cc:107] Waiting for socket to connect
0810 11:35:23.431110 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.254:42587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":472,"thread_start_us":340,"threads_started":1}
I20260810 11:35:23.921389 25836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:23.920505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:42587 (local address 127.8.18.193:59169)
0810 11:35:23.921088 (+ 583us) negotiation.cc:107] Waiting for socket to connect
0810 11:35:23.921204 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.254:42587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":445,"thread_start_us":330,"threads_started":1}
I20260810 11:35:23.939349 25837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:23.938453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:42587 (local address 127.8.18.195:43461)
0810 11:35:23.939040 (+ 587us) negotiation.cc:107] Waiting for socket to connect
0810 11:35:23.939156 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.254:42587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":433,"thread_start_us":271,"threads_started":1}
I20260810 11:35:24.433658 25838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:24.432616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:42587 (local address 127.8.18.194:48715)
0810 11:35:24.433229 (+ 613us) negotiation.cc:107] Waiting for socket to connect
0810 11:35:24.433353 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.254:42587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":458,"thread_start_us":309,"threads_started":1}
I20260810 11:35:24.906670 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 25348
I20260810 11:35:24.923774 25840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:24.922676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:42587 (local address 127.8.18.193:37005)
0810 11:35:24.923332 (+ 656us) negotiation.cc:107] Waiting for socket to connect
0810 11:35:24.923495 (+ 163us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.254:42587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":538,"thread_start_us":267,"threads_started":1}
I20260810 11:35:24.936578 25839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:24.908329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33368 (local address 127.8.18.193:43511)
0810 11:35:24.908984 (+ 655us) server_negotiation.cc:206] Beginning negotiation
0810 11:35:24.909000 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0810 11:35:24.909048 (+ 48us) server_negotiation.cc:407] Connection header received
0810 11:35:24.910810 (+ 1762us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:35:24.910823 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:35:24.911306 (+ 483us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:35:24.911659 (+ 353us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:35:24.913561 (+ 1902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:24.914912 (+ 1351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:24.916426 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:24.917315 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:24.917609 (+ 294us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:35:24.928336 (+ 10727us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:35:24.928347 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:35:24.928368 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:35:24.928424 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:35:24.932295 (+ 3871us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:35:24.933229 (+ 934us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:35:24.933239 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:35:24.933248 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:35:24.933581 (+ 333us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:35:24.934086 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:35:24.934092 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:35:24.934095 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:35:24.934997 (+ 902us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:35:24.935271 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0810 11:35:24.935735 (+ 464us) server_negotiation.cc:299] Negotiation successful
0810 11:35:24.936243 (+ 508us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":480,"thread_start_us":328,"threads_started":1}
I20260810 11:35:24.938524 25513 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:35:24.941736 25842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:24.940716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:42587 (local address 127.8.18.195:57921)
0810 11:35:24.941316 (+ 600us) negotiation.cc:107] Waiting for socket to connect
0810 11:35:24.941475 (+ 159us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.254:42587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":442,"thread_start_us":305,"threads_started":1}
I20260810 11:35:25.028654 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 25348
I20260810 11:35:25.051854 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 25521
I20260810 11:35:25.072321 25843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:25.052905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56038 (local address 127.8.18.194:34511)
0810 11:35:25.053568 (+ 663us) server_negotiation.cc:206] Beginning negotiation
0810 11:35:25.053584 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0810 11:35:25.053626 (+ 42us) server_negotiation.cc:407] Connection header received
0810 11:35:25.055307 (+ 1681us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:35:25.055319 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:35:25.055771 (+ 452us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:35:25.056156 (+ 385us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:35:25.057909 (+ 1753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:25.059202 (+ 1293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:25.060553 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:25.061318 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:25.061589 (+ 271us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:35:25.064650 (+ 3061us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:35:25.064661 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:35:25.064681 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:35:25.064733 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:35:25.068216 (+ 3483us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:35:25.069137 (+ 921us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:35:25.069146 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:35:25.069154 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:35:25.069378 (+ 224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:35:25.069895 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:35:25.069901 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:35:25.069905 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:35:25.070544 (+ 639us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:35:25.070903 (+ 359us) server_negotiation.cc:1035] Waiting for connection context
0810 11:35:25.071496 (+ 593us) server_negotiation.cc:299] Negotiation successful
0810 11:35:25.072001 (+ 505us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":423,"thread_start_us":285,"threads_started":1}
I20260810 11:35:25.074096 25646 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:35:25.171451 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 25521
I20260810 11:35:25.192817 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 25654
I20260810 11:35:25.216611 25845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:25.194192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44358 (local address 127.8.18.195:40573)
0810 11:35:25.194952 (+ 760us) server_negotiation.cc:206] Beginning negotiation
0810 11:35:25.194966 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 11:35:25.195010 (+ 44us) server_negotiation.cc:407] Connection header received
0810 11:35:25.196719 (+ 1709us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:35:25.196730 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:35:25.197186 (+ 456us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:35:25.197578 (+ 392us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:35:25.199758 (+ 2180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:25.201103 (+ 1345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:25.203346 (+ 2243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:25.204157 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:25.204321 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:35:25.208554 (+ 4233us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:35:25.208563 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:35:25.208581 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:35:25.208638 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:35:25.212140 (+ 3502us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:35:25.213486 (+ 1346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:35:25.213495 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:35:25.213503 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:35:25.213726 (+ 223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:35:25.214227 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:35:25.214230 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:35:25.214232 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:35:25.214856 (+ 624us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:35:25.215106 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0810 11:35:25.215727 (+ 621us) server_negotiation.cc:299] Negotiation successful
0810 11:35:25.216236 (+ 509us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":585,"thread_start_us":395,"threads_started":1}
I20260810 11:35:25.218356 25784 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:35:25.314881 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 25654
2026-08-10T11:35:25Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (65572 ms)
[----------] 30 tests from SecurityITest (323251 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786361401922362-8267-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 11:35:25 dist-test-slave-srgv krb5kdc[25849](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 11:35:25 dist-test-slave-srgv krb5kdc[25849](info): set up 2 sockets
Aug 10 11:35:25 dist-test-slave-srgv krb5kdc[25849](info): commencing operation
krb5kdc: starting...
W20260810 11:35:26.460985 8267 mini_kdc.cc:121] Time spent starting KDC: real 1.101s user 0.000s sys 0.043s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786361401922362-8267-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786361401922362-8267-0/krb5kdc/test-user.keytab.
Aug 10 11:35:26 dist-test-slave-srgv krb5kdc[25849](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361726, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T11:35:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T11:35:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.18.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.18.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.18.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.254.keytab.
Entry for principal kudu/127.8.18.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.254.keytab.
I20260810 11:35:26.785000 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786361401922362-8267-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786361401922362-8267-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786361401922362-8267-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.18.254:38989
--webserver_interface=127.8.18.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.254.keytab
--principal=kudu/127.8.18.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.18.212:44677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.18.254:38989
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786361401922362-8267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786361401922362-8267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786361401922362-8267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:35:27.157756 25862 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:35:27.158172 25862 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:35:27.158387 25862 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:35:27.166172 25862 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 11:35:27.166291 25862 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:35:27.166407 25862 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:35:27.166509 25862 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 11:35:27.166601 25862 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 11:35:27.177609 25862 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.18.212:44677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786361401922362-8267-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786361401922362-8267-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.18.254:38989
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.254.keytab
--principal=kudu/127.8.18.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.18.254:38989
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.18.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786361401922362-8267-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:35:27.179081 25862 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:35:27.180437 25862 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:35:27.193869 25867 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:35:27.193833 25868 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:35:27.193784 25870 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:35:27.194674 25862 server_base.cc:1034] running on GCE node
I20260810 11:35:27.195672 25862 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:35:27.197356 25862 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:35:27.198673 25862 hybrid_clock.cc:648] HybridClock initialized: now 1786361727198625 us; error 40 us; skew 500 ppm
I20260810 11:35:27.199092 25862 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:35:27 dist-test-slave-srgv krb5kdc[25849](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361727, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:35:27.202908 25862 init.cc:377] Logged in from keytab as kudu/127.8.18.254@KRBTEST.COM (short username kudu)
I20260810 11:35:27.204811 25862 webserver.cc:460] Webserver started at http://127.8.18.254:42105/ using document root <none> and password file <none>
I20260810 11:35:27.205708 25862 fs_manager.cc:359] Metadata directory not provided
I20260810 11:35:27.205876 25862 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:35:27.206223 25862 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:35:27.209554 25862 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786361401922362-8267-0/minicluster-data/master-0/data/instance:
uuid: "73ec08971ce84042909f6b92427ef790"
format_stamp: "Formatted at 2026-08-10 11:35:27 on dist-test-slave-srgv"
server_key: "45d62228d908956878ffdd3662fa556a"
server_key_iv: "91bba969621c9838ba21f73ce22ebd2c"
server_key_version: "encryptionkey@0"
I20260810 11:35:27.210438 25862 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786361401922362-8267-0/minicluster-data/master-0/wal/instance:
uuid: "73ec08971ce84042909f6b92427ef790"
format_stamp: "Formatted at 2026-08-10 11:35:27 on dist-test-slave-srgv"
server_key: "45d62228d908956878ffdd3662fa556a"
server_key_iv: "91bba969621c9838ba21f73ce22ebd2c"
server_key_version: "encryptionkey@0"
I20260810 11:35:27.216282 25862 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260810 11:35:27.220359 25877 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:35:27.222129 25862 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 11:35:27.222402 25862 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786361401922362-8267-0/minicluster-data/master-0/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786361401922362-8267-0/minicluster-data/master-0/wal
uuid: "73ec08971ce84042909f6b92427ef790"
format_stamp: "Formatted at 2026-08-10 11:35:27 on dist-test-slave-srgv"
server_key: "45d62228d908956878ffdd3662fa556a"
server_key_iv: "91bba969621c9838ba21f73ce22ebd2c"
server_key_version: "encryptionkey@0"
I20260810 11:35:27.222697 25862 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786361401922362-8267-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786361401922362-8267-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786361401922362-8267-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:35:27.261744 25862 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:35:27.266955 25862 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:35:27.267335 25862 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:35:27.290242 25862 rpc_server.cc:307] RPC server started. Bound to: 127.8.18.254:38989
I20260810 11:35:27.290272 25930 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.18.254:38989 every 8 connection(s)
I20260810 11:35:27.292066 25862 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
I20260810 11:35:27.296684 25931 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:35:27.298332 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 25862
I20260810 11:35:27.298595 8267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786361401922362-8267-0/minicluster-data/master-0/wal/instance
I20260810 11:35:27.299331 8267 external_mini_cluster.cc:1428] Setting key 6ffc0802f322bf4252d5f71c48d07f40
I20260810 11:35:27.312798 25931 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 73ec08971ce84042909f6b92427ef790: Bootstrap starting.
Aug 10 11:35:27 dist-test-slave-srgv krb5kdc[25849](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361726, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.18.254@KRBTEST.COM
I20260810 11:35:27.316687 25931 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 73ec08971ce84042909f6b92427ef790: Neither blocks nor log segments found. Creating new log.
I20260810 11:35:27.318056 25931 log.cc:824] T 00000000000000000000000000000000 P 73ec08971ce84042909f6b92427ef790: Log is configured to *not* fsync() on all Append() calls
I20260810 11:35:27.322528 25931 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 73ec08971ce84042909f6b92427ef790: No bootstrap required, opened a new log
I20260810 11:35:27.328565 25934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:27.301712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60724 (local address 127.8.18.254:38989)
0810 11:35:27.302696 (+ 984us) server_negotiation.cc:206] Beginning negotiation
0810 11:35:27.302713 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0810 11:35:27.302781 (+ 68us) server_negotiation.cc:407] Connection header received
0810 11:35:27.304737 (+ 1956us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:35:27.304772 (+ 35us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:35:27.305382 (+ 610us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:35:27.305896 (+ 514us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:35:27.307727 (+ 1831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:27.309984 (+ 2257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:27.311622 (+ 1638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:27.312781 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:27.312997 (+ 216us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:35:27.318180 (+ 5183us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:35:27.318195 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:35:27.318219 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:35:27.318287 (+ 68us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:35:27.322830 (+ 4543us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:35:27.323866 (+ 1036us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:35:27.323878 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:35:27.323888 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:35:27.324177 (+ 289us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:35:27.324679 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:35:27.324687 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:35:27.324695 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:35:27.325585 (+ 890us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:35:27.325941 (+ 356us) server_negotiation.cc:1035] Waiting for connection context
0810 11:35:27.326414 (+ 473us) server_negotiation.cc:299] Negotiation successful
0810 11:35:27.326949 (+ 535us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":703,"thread_start_us":431,"threads_started":1}
I20260810 11:35:27.333006 25931 raft_consensus.cc:348] T 00000000000000000000000000000000 P 73ec08971ce84042909f6b92427ef790 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73ec08971ce84042909f6b92427ef790" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 38989 } }
I20260810 11:35:27.333451 25931 raft_consensus.cc:374] T 00000000000000000000000000000000 P 73ec08971ce84042909f6b92427ef790 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:35:27.333592 25931 raft_consensus.cc:729] T 00000000000000000000000000000000 P 73ec08971ce84042909f6b92427ef790 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73ec08971ce84042909f6b92427ef790, State: Initialized, Role: FOLLOWER
I20260810 11:35:27.334214 25931 consensus_queue.cc:260] T 00000000000000000000000000000000 P 73ec08971ce84042909f6b92427ef790 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73ec08971ce84042909f6b92427ef790" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 38989 } }
I20260810 11:35:27.334471 25931 raft_consensus.cc:388] T 00000000000000000000000000000000 P 73ec08971ce84042909f6b92427ef790 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 11:35:27.334605 25931 raft_consensus.cc:482] T 00000000000000000000000000000000 P 73ec08971ce84042909f6b92427ef790 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 11:35:27.334792 25931 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 73ec08971ce84042909f6b92427ef790 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:35:27.337230 25931 raft_consensus.cc:504] T 00000000000000000000000000000000 P 73ec08971ce84042909f6b92427ef790 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73ec08971ce84042909f6b92427ef790" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 38989 } }
I20260810 11:35:27.337908 25931 leader_election.cc:302] T 00000000000000000000000000000000 P 73ec08971ce84042909f6b92427ef790 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 73ec08971ce84042909f6b92427ef790; no voters:
I20260810 11:35:27.338605 25931 leader_election.cc:288] T 00000000000000000000000000000000 P 73ec08971ce84042909f6b92427ef790 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 11:35:27.338794 25936 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 73ec08971ce84042909f6b92427ef790 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:35:27.339653 25936 raft_consensus.cc:686] T 00000000000000000000000000000000 P 73ec08971ce84042909f6b92427ef790 [term 1 LEADER]: Becoming Leader. State: Replica: 73ec08971ce84042909f6b92427ef790, State: Running, Role: LEADER
I20260810 11:35:27.340453 25936 consensus_queue.cc:237] T 00000000000000000000000000000000 P 73ec08971ce84042909f6b92427ef790 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73ec08971ce84042909f6b92427ef790" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 38989 } }
I20260810 11:35:27.341045 25931 sys_catalog.cc:564] T 00000000000000000000000000000000 P 73ec08971ce84042909f6b92427ef790 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 11:35:27.344030 25937 sys_catalog.cc:455] T 00000000000000000000000000000000 P 73ec08971ce84042909f6b92427ef790 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "73ec08971ce84042909f6b92427ef790" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73ec08971ce84042909f6b92427ef790" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 38989 } } }
I20260810 11:35:27.344486 25937 sys_catalog.cc:458] T 00000000000000000000000000000000 P 73ec08971ce84042909f6b92427ef790 [sys.catalog]: This master's current role is: LEADER
I20260810 11:35:27.344810 25938 sys_catalog.cc:455] T 00000000000000000000000000000000 P 73ec08971ce84042909f6b92427ef790 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 73ec08971ce84042909f6b92427ef790. Latest consensus state: current_term: 1 leader_uuid: "73ec08971ce84042909f6b92427ef790" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73ec08971ce84042909f6b92427ef790" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 38989 } } }
I20260810 11:35:27.345135 25938 sys_catalog.cc:458] T 00000000000000000000000000000000 P 73ec08971ce84042909f6b92427ef790 [sys.catalog]: This master's current role is: LEADER
I20260810 11:35:27.347004 25943 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 11:35:27.353133 25943 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 11:35:27.362812 25943 catalog_manager.cc:1347] Generated new cluster ID: 91865f32f19d41db86b042dcaaad296f
I20260810 11:35:27.362963 25943 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 11:35:27.376150 25943 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 11:35:27.377753 25943 catalog_manager.cc:1504] Loading token signing keys...
I20260810 11:35:27.388772 25943 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 73ec08971ce84042909f6b92427ef790: Generated new TSK 0
I20260810 11:35:27.389853 25943 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.18.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.18.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.18.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.193.keytab.
Entry for principal kudu/127.8.18.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.193.keytab.
I20260810 11:35:27.477576 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786361401922362-8267-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786361401922362-8267-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786361401922362-8267-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786361401922362-8267-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.18.193:0
--local_ip_for_outbound_sockets=127.8.18.193
--webserver_interface=127.8.18.193
--webserver_port=0
--tserver_master_addrs=127.8.18.254:38989
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.193.keytab
--principal=kudu/127.8.18.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.18.212:44677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786361401922362-8267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786361401922362-8267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786361401922362-8267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:35:27.853013 25957 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:35:27.853361 25957 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:35:27.853562 25957 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:35:27.861860 25957 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:35:27.861995 25957 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:35:27.862120 25957 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.18.193
I20260810 11:35:27.872641 25957 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.18.212:44677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786361401922362-8267-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786361401922362-8267-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.193.keytab
--principal=kudu/127.8.18.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.18.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786361401922362-8267-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.18.193
--webserver_port=0
--tserver_master_addrs=127.8.18.254:38989
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.18.193
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786361401922362-8267-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:35:27.874191 25957 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:35:27.875528 25957 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:35:27.890339 25963 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:35:27.890580 25964 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:35:27.891407 25966 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:35:27.892112 25957 server_base.cc:1034] running on GCE node
I20260810 11:35:27.892815 25957 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:35:27.893972 25957 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:35:27.895262 25957 hybrid_clock.cc:648] HybridClock initialized: now 1786361727895217 us; error 47 us; skew 500 ppm
I20260810 11:35:27.895632 25957 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:35:27 dist-test-slave-srgv krb5kdc[25849](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361727, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:35:27.899356 25957 init.cc:377] Logged in from keytab as kudu/127.8.18.193@KRBTEST.COM (short username kudu)
I20260810 11:35:27.901106 25957 webserver.cc:460] Webserver started at http://127.8.18.193:37967/ using document root <none> and password file <none>
I20260810 11:35:27.901945 25957 fs_manager.cc:359] Metadata directory not provided
I20260810 11:35:27.902129 25957 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:35:27.902486 25957 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:35:27.905755 25957 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786361401922362-8267-0/minicluster-data/ts-0/data/instance:
uuid: "0ff2be4158114536973e89ffaf108d7a"
format_stamp: "Formatted at 2026-08-10 11:35:27 on dist-test-slave-srgv"
server_key: "c1133b9d8870dfebaac00b5e19ada1da"
server_key_iv: "886cd84bea7d743cd5e10d8e713dae7d"
server_key_version: "encryptionkey@0"
I20260810 11:35:27.906658 25957 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786361401922362-8267-0/minicluster-data/ts-0/wal/instance:
uuid: "0ff2be4158114536973e89ffaf108d7a"
format_stamp: "Formatted at 2026-08-10 11:35:27 on dist-test-slave-srgv"
server_key: "c1133b9d8870dfebaac00b5e19ada1da"
server_key_iv: "886cd84bea7d743cd5e10d8e713dae7d"
server_key_version: "encryptionkey@0"
I20260810 11:35:27.912532 25957 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260810 11:35:27.917152 25973 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:35:27.918835 25957 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 11:35:27.919078 25957 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786361401922362-8267-0/minicluster-data/ts-0/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786361401922362-8267-0/minicluster-data/ts-0/wal
uuid: "0ff2be4158114536973e89ffaf108d7a"
format_stamp: "Formatted at 2026-08-10 11:35:27 on dist-test-slave-srgv"
server_key: "c1133b9d8870dfebaac00b5e19ada1da"
server_key_iv: "886cd84bea7d743cd5e10d8e713dae7d"
server_key_version: "encryptionkey@0"
I20260810 11:35:27.919370 25957 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786361401922362-8267-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786361401922362-8267-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786361401922362-8267-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:35:27.945393 25957 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:35:27.952149 25957 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:35:27.952538 25957 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:35:27.953715 25957 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:35:27.955169 25957 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:35:27.955248 25957 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:35:27.955394 25957 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:35:27.955454 25957 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:35:27.992353 25957 rpc_server.cc:307] RPC server started. Bound to: 127.8.18.193:45573
I20260810 11:35:27.992551 26096 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.18.193:45573 every 8 connection(s)
I20260810 11:35:27.993845 25957 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786361401922362-8267-0/minicluster-data/ts-0/data/info.pb
I20260810 11:35:28.001883 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 25957
I20260810 11:35:28.002127 8267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786361401922362-8267-0/minicluster-data/ts-0/wal/instance
I20260810 11:35:28.002837 8267 external_mini_cluster.cc:1428] Setting key eb3911b7a25af5c180ea217433878bf0
Aug 10 11:35:28 dist-test-slave-srgv krb5kdc[25849](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361727, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.193@KRBTEST.COM for kudu/127.8.18.254@KRBTEST.COM
I20260810 11:35:28.017602 26099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:27.996819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.193:42793 (local address 127.8.18.254:38989)
0810 11:35:27.997329 (+ 510us) server_negotiation.cc:206] Beginning negotiation
0810 11:35:27.997334 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:35:27.998457 (+ 1123us) server_negotiation.cc:407] Connection header received
0810 11:35:28.000440 (+ 1983us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:35:28.000447 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:35:28.000564 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:35:28.000687 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:35:28.002555 (+ 1868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:28.003357 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:28.005324 (+ 1967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:28.005862 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:28.006004 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:35:28.010084 (+ 4080us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:35:28.010094 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:35:28.010101 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:35:28.010152 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:35:28.014030 (+ 3878us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:35:28.014903 (+ 873us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:35:28.014912 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:35:28.014918 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:35:28.015089 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:35:28.015548 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:35:28.015556 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:35:28.015561 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:35:28.015961 (+ 400us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:35:28.016140 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0810 11:35:28.016980 (+ 840us) server_negotiation.cc:299] Negotiation successful
0810 11:35:28.017235 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":390,"thread_start_us":274,"threads_started":1}
I20260810 11:35:28.019227 26100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:27.997464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:38989 (local address 127.8.18.193:42793)
0810 11:35:27.998275 (+ 811us) negotiation.cc:107] Waiting for socket to connect
0810 11:35:27.998321 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0810 11:35:28.000141 (+ 1820us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:35:28.000944 (+ 803us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:35:28.000952 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:35:28.001653 (+ 701us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:35:28.002366 (+ 713us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:35:28.002387 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:35:28.003539 (+ 1152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:35:28.003543 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:35:28.005170 (+ 1627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:35:28.005183 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:35:28.006105 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:35:28.006115 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:35:28.006362 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:35:28.007105 (+ 743us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:35:28.007129 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:35:28.009810 (+ 2681us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:35:28.014215 (+ 4405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:35:28.014225 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:35:28.014240 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:35:28.014742 (+ 502us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:35:28.015211 (+ 469us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:35:28.015214 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:35:28.015216 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:35:28.015416 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:35:28.016149 (+ 733us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:35:28.016156 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:35:28.016521 (+ 365us) client_negotiation.cc:769] Sending connection context
0810 11:35:28.016806 (+ 285us) client_negotiation.cc:240] Negotiation successful
0810 11:35:28.017580 (+ 774us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":548,"thread_start_us":319,"threads_started":1}
I20260810 11:35:28.021034 26097 heartbeater.cc:347] Connected to a master server at 127.8.18.254:38989
I20260810 11:35:28.021466 26097 heartbeater.cc:464] Registering TS with master...
I20260810 11:35:28.022323 26097 heartbeater.cc:511] Master 127.8.18.254:38989 requested a full tablet report, sending...
I20260810 11:35:28.024951 25895 ts_manager.cc:194] Registered new tserver with Master: 0ff2be4158114536973e89ffaf108d7a (127.8.18.193:45573)
I20260810 11:35:28.026604 25895 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.18.193@KRBTEST.COM'} at 127.8.18.193:42793
WARNING: no policy specified for kudu/127.8.18.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.18.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.18.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.194.keytab.
Entry for principal kudu/127.8.18.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.194.keytab.
I20260810 11:35:28.074821 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786361401922362-8267-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786361401922362-8267-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786361401922362-8267-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786361401922362-8267-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.18.194:0
--local_ip_for_outbound_sockets=127.8.18.194
--webserver_interface=127.8.18.194
--webserver_port=0
--tserver_master_addrs=127.8.18.254:38989
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.194.keytab
--principal=kudu/127.8.18.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.18.212:44677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786361401922362-8267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786361401922362-8267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786361401922362-8267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:35:28.460457 26104 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:35:28.460819 26104 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:35:28.461004 26104 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:35:28.468664 26104 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:35:28.468853 26104 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:35:28.469065 26104 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.18.194
I20260810 11:35:28.479733 26104 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.18.212:44677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786361401922362-8267-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786361401922362-8267-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.194.keytab
--principal=kudu/127.8.18.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.18.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786361401922362-8267-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.18.194
--webserver_port=0
--tserver_master_addrs=127.8.18.254:38989
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.18.194
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786361401922362-8267-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:35:28.481241 26104 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:35:28.482748 26104 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:35:28.497570 26109 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:35:28.497545 26112 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:35:28.497535 26110 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:35:28.498971 26104 server_base.cc:1034] running on GCE node
I20260810 11:35:28.499701 26104 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:35:28.500751 26104 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:35:28.501995 26104 hybrid_clock.cc:648] HybridClock initialized: now 1786361728501944 us; error 45 us; skew 500 ppm
I20260810 11:35:28.502378 26104 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:35:28 dist-test-slave-srgv krb5kdc[25849](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361728, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:35:28.505932 26104 init.cc:377] Logged in from keytab as kudu/127.8.18.194@KRBTEST.COM (short username kudu)
I20260810 11:35:28.507786 26104 webserver.cc:460] Webserver started at http://127.8.18.194:42665/ using document root <none> and password file <none>
I20260810 11:35:28.508564 26104 fs_manager.cc:359] Metadata directory not provided
I20260810 11:35:28.508723 26104 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:35:28.509084 26104 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:35:28.512348 26104 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786361401922362-8267-0/minicluster-data/ts-1/data/instance:
uuid: "62fc62a2e30244009741c41455bfe3c9"
format_stamp: "Formatted at 2026-08-10 11:35:28 on dist-test-slave-srgv"
server_key: "dc43e2950801409e99d83a384ad77ee8"
server_key_iv: "52220ba644c621b0c30f70b88b516b19"
server_key_version: "encryptionkey@0"
I20260810 11:35:28.513257 26104 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786361401922362-8267-0/minicluster-data/ts-1/wal/instance:
uuid: "62fc62a2e30244009741c41455bfe3c9"
format_stamp: "Formatted at 2026-08-10 11:35:28 on dist-test-slave-srgv"
server_key: "dc43e2950801409e99d83a384ad77ee8"
server_key_iv: "52220ba644c621b0c30f70b88b516b19"
server_key_version: "encryptionkey@0"
I20260810 11:35:28.519307 26104 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260810 11:35:28.523945 26119 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:35:28.525702 26104 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260810 11:35:28.525910 26104 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786361401922362-8267-0/minicluster-data/ts-1/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786361401922362-8267-0/minicluster-data/ts-1/wal
uuid: "62fc62a2e30244009741c41455bfe3c9"
format_stamp: "Formatted at 2026-08-10 11:35:28 on dist-test-slave-srgv"
server_key: "dc43e2950801409e99d83a384ad77ee8"
server_key_iv: "52220ba644c621b0c30f70b88b516b19"
server_key_version: "encryptionkey@0"
I20260810 11:35:28.526170 26104 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786361401922362-8267-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786361401922362-8267-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786361401922362-8267-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:35:28.542201 26104 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:35:28.547611 26104 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:35:28.548004 26104 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:35:28.549151 26104 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:35:28.550742 26104 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:35:28.550856 26104 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:35:28.551020 26104 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:35:28.551110 26104 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:35:28.587555 26104 rpc_server.cc:307] RPC server started. Bound to: 127.8.18.194:38411
I20260810 11:35:28.587616 26233 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.18.194:38411 every 8 connection(s)
I20260810 11:35:28.588990 26104 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786361401922362-8267-0/minicluster-data/ts-1/data/info.pb
I20260810 11:35:28.599399 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 26104
I20260810 11:35:28.599602 8267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786361401922362-8267-0/minicluster-data/ts-1/wal/instance
I20260810 11:35:28.600126 8267 external_mini_cluster.cc:1428] Setting key f669c8bf222b6ab4b3f2101260fd54c2
Aug 10 11:35:28 dist-test-slave-srgv krb5kdc[25849](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361728, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.194@KRBTEST.COM for kudu/127.8.18.254@KRBTEST.COM
I20260810 11:35:28.610971 26236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:28.591808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.194:50917 (local address 127.8.18.254:38989)
0810 11:35:28.592285 (+ 477us) server_negotiation.cc:206] Beginning negotiation
0810 11:35:28.592290 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:35:28.593255 (+ 965us) server_negotiation.cc:407] Connection header received
0810 11:35:28.595312 (+ 2057us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:35:28.595318 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:35:28.595430 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:35:28.595560 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:35:28.597218 (+ 1658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:28.598040 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:28.599920 (+ 1880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:28.600433 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:28.600584 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:35:28.604735 (+ 4151us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:35:28.604741 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:35:28.604745 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:35:28.604782 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:35:28.607564 (+ 2782us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:35:28.608394 (+ 830us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:35:28.608402 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:35:28.608405 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:35:28.608537 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:35:28.609014 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:35:28.609019 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:35:28.609023 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:35:28.609334 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:35:28.609550 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0810 11:35:28.610332 (+ 782us) server_negotiation.cc:299] Negotiation successful
0810 11:35:28.610588 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":360,"thread_start_us":255,"threads_started":1}
I20260810 11:35:28.612036 26237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:28.592310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:38989 (local address 127.8.18.194:50917)
0810 11:35:28.593044 (+ 734us) negotiation.cc:107] Waiting for socket to connect
0810 11:35:28.593088 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 11:35:28.594991 (+ 1903us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:35:28.595823 (+ 832us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:35:28.595832 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:35:28.596417 (+ 585us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:35:28.597018 (+ 601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:35:28.597037 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:35:28.598230 (+ 1193us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:35:28.598234 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:35:28.599765 (+ 1531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:35:28.599779 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:35:28.600693 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:35:28.600701 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:35:28.600958 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:35:28.601881 (+ 923us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:35:28.601914 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:35:28.604461 (+ 2547us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:35:28.607754 (+ 3293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:35:28.607765 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:35:28.607791 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:35:28.608259 (+ 468us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:35:28.608659 (+ 400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:35:28.608665 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:35:28.608669 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:35:28.608891 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:35:28.609531 (+ 640us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:35:28.609540 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:35:28.609940 (+ 400us) client_negotiation.cc:769] Sending connection context
0810 11:35:28.610229 (+ 289us) client_negotiation.cc:240] Negotiation successful
0810 11:35:28.610835 (+ 606us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":503,"thread_start_us":304,"threads_started":1}
I20260810 11:35:28.613618 26234 heartbeater.cc:347] Connected to a master server at 127.8.18.254:38989
I20260810 11:35:28.614001 26234 heartbeater.cc:464] Registering TS with master...
I20260810 11:35:28.614985 26234 heartbeater.cc:511] Master 127.8.18.254:38989 requested a full tablet report, sending...
I20260810 11:35:28.616667 25895 ts_manager.cc:194] Registered new tserver with Master: 62fc62a2e30244009741c41455bfe3c9 (127.8.18.194:38411)
I20260810 11:35:28.617648 25895 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.18.194@KRBTEST.COM'} at 127.8.18.194:50917
WARNING: no policy specified for kudu/127.8.18.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.18.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.18.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.195.keytab.
Entry for principal kudu/127.8.18.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.195.keytab.
I20260810 11:35:28.669003 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786361401922362-8267-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786361401922362-8267-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786361401922362-8267-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786361401922362-8267-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.18.195:0
--local_ip_for_outbound_sockets=127.8.18.195
--webserver_interface=127.8.18.195
--webserver_port=0
--tserver_master_addrs=127.8.18.254:38989
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.195.keytab
--principal=kudu/127.8.18.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.18.212:44677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786361401922362-8267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786361401922362-8267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786361401922362-8267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:35:29.029804 26240 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
I20260810 11:35:29.030261 26097 heartbeater.cc:503] Master 127.8.18.254:38989 was elected leader, sending a full tablet report...
W20260810 11:35:29.030349 26240 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:35:29.030524 26240 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:35:29.038766 26240 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:35:29.038913 26240 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:35:29.039103 26240 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.18.195
I20260810 11:35:29.050431 26240 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.18.212:44677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786361401922362-8267-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786361401922362-8267-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.195.keytab
--principal=kudu/127.8.18.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.18.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786361401922362-8267-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.18.195
--webserver_port=0
--tserver_master_addrs=127.8.18.254:38989
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.18.195
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786361401922362-8267-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:35:29.051863 26240 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:35:29.053274 26240 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:35:29.067658 26245 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:35:29.067658 26247 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:35:29.067704 26249 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:35:29.068827 26240 server_base.cc:1034] running on GCE node
I20260810 11:35:29.069591 26240 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:35:29.070659 26240 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:35:29.071936 26240 hybrid_clock.cc:648] HybridClock initialized: now 1786361729071864 us; error 67 us; skew 500 ppm
I20260810 11:35:29.072314 26240 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:35:29 dist-test-slave-srgv krb5kdc[25849](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361729, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:35:29.076251 26240 init.cc:377] Logged in from keytab as kudu/127.8.18.195@KRBTEST.COM (short username kudu)
I20260810 11:35:29.078060 26240 webserver.cc:460] Webserver started at http://127.8.18.195:39621/ using document root <none> and password file <none>
I20260810 11:35:29.078810 26240 fs_manager.cc:359] Metadata directory not provided
I20260810 11:35:29.078928 26240 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:35:29.079227 26240 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:35:29.082530 26240 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786361401922362-8267-0/minicluster-data/ts-2/data/instance:
uuid: "39130be9337646e0b2b1b0f5e025e1c7"
format_stamp: "Formatted at 2026-08-10 11:35:29 on dist-test-slave-srgv"
server_key: "84d55cfa6f721910fbc97768aec22c11"
server_key_iv: "27df0b3a868091e4847d43d0e1b36a1a"
server_key_version: "encryptionkey@0"
I20260810 11:35:29.083431 26240 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786361401922362-8267-0/minicluster-data/ts-2/wal/instance:
uuid: "39130be9337646e0b2b1b0f5e025e1c7"
format_stamp: "Formatted at 2026-08-10 11:35:29 on dist-test-slave-srgv"
server_key: "84d55cfa6f721910fbc97768aec22c11"
server_key_iv: "27df0b3a868091e4847d43d0e1b36a1a"
server_key_version: "encryptionkey@0"
I20260810 11:35:29.089591 26240 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260810 11:35:29.093727 26256 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:35:29.095358 26240 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 11:35:29.095534 26240 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786361401922362-8267-0/minicluster-data/ts-2/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786361401922362-8267-0/minicluster-data/ts-2/wal
uuid: "39130be9337646e0b2b1b0f5e025e1c7"
format_stamp: "Formatted at 2026-08-10 11:35:29 on dist-test-slave-srgv"
server_key: "84d55cfa6f721910fbc97768aec22c11"
server_key_iv: "27df0b3a868091e4847d43d0e1b36a1a"
server_key_version: "encryptionkey@0"
I20260810 11:35:29.095742 26240 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786361401922362-8267-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786361401922362-8267-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786361401922362-8267-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:35:29.121901 26240 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:35:29.127074 26240 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:35:29.127460 26240 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:35:29.128602 26240 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:35:29.130239 26240 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:35:29.130396 26240 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:35:29.130554 26240 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:35:29.130609 26240 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:35:29.168404 26240 rpc_server.cc:307] RPC server started. Bound to: 127.8.18.195:40587
I20260810 11:35:29.168502 26376 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.18.195:40587 every 8 connection(s)
I20260810 11:35:29.169883 26240 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786361401922362-8267-0/minicluster-data/ts-2/data/info.pb
I20260810 11:35:29.172152 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 26240
I20260810 11:35:29.172374 8267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786361401922362-8267-0/minicluster-data/ts-2/wal/instance
I20260810 11:35:29.172876 8267 external_mini_cluster.cc:1428] Setting key aeff76d04558333ad1e35d4284e8063b
Aug 10 11:35:29 dist-test-slave-srgv krb5kdc[25849](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361729, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.195@KRBTEST.COM for kudu/127.8.18.254@KRBTEST.COM
I20260810 11:35:29.193876 26379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:29.173118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.195:37391 (local address 127.8.18.254:38989)
0810 11:35:29.173780 (+ 662us) server_negotiation.cc:206] Beginning negotiation
0810 11:35:29.173789 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 11:35:29.175012 (+ 1223us) server_negotiation.cc:407] Connection header received
0810 11:35:29.177915 (+ 2903us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:35:29.177920 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:35:29.178039 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:35:29.178160 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:35:29.179870 (+ 1710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:29.180741 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:29.182541 (+ 1800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:29.182959 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:29.183130 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:35:29.187260 (+ 4130us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:35:29.187266 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:35:29.187269 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:35:29.187309 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:35:29.190456 (+ 3147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:35:29.191339 (+ 883us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:35:29.191344 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:35:29.191346 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:35:29.191478 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:35:29.192073 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:35:29.192077 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:35:29.192079 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:35:29.192395 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:35:29.192532 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0810 11:35:29.193293 (+ 761us) server_negotiation.cc:299] Negotiation successful
0810 11:35:29.193551 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":502,"thread_start_us":327,"threads_started":1}
I20260810 11:35:29.194844 26380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:29.173705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:38989 (local address 127.8.18.195:37391)
0810 11:35:29.174821 (+ 1116us) negotiation.cc:107] Waiting for socket to connect
0810 11:35:29.174865 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 11:35:29.177621 (+ 2756us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:35:29.178432 (+ 811us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:35:29.178442 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:35:29.179032 (+ 590us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:35:29.179670 (+ 638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:35:29.179691 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:35:29.180935 (+ 1244us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:35:29.180939 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:35:29.182354 (+ 1415us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:35:29.182376 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:35:29.183250 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:35:29.183258 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:35:29.183529 (+ 271us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:35:29.184317 (+ 788us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:35:29.184342 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:35:29.186965 (+ 2623us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:35:29.190647 (+ 3682us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:35:29.190658 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:35:29.190678 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:35:29.191186 (+ 508us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:35:29.191607 (+ 421us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:35:29.191612 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:35:29.191616 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:35:29.191924 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:35:29.192537 (+ 613us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:35:29.192544 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:35:29.192916 (+ 372us) client_negotiation.cc:769] Sending connection context
0810 11:35:29.193167 (+ 251us) client_negotiation.cc:240] Negotiation successful
0810 11:35:29.193707 (+ 540us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":754,"thread_start_us":449,"threads_started":1}
I20260810 11:35:29.196419 26377 heartbeater.cc:347] Connected to a master server at 127.8.18.254:38989
I20260810 11:35:29.196702 26377 heartbeater.cc:464] Registering TS with master...
I20260810 11:35:29.197388 26377 heartbeater.cc:511] Master 127.8.18.254:38989 requested a full tablet report, sending...
I20260810 11:35:29.199028 25895 ts_manager.cc:194] Registered new tserver with Master: 39130be9337646e0b2b1b0f5e025e1c7 (127.8.18.195:40587)
I20260810 11:35:29.199975 25895 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.18.195@KRBTEST.COM'} at 127.8.18.195:37391
I20260810 11:35:29.203092 8267 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 11:35:29.222134 26379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:29.208151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60728 (local address 127.8.18.254:38989)
0810 11:35:29.208445 (+ 294us) server_negotiation.cc:206] Beginning negotiation
0810 11:35:29.208453 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:35:29.208761 (+ 308us) server_negotiation.cc:407] Connection header received
0810 11:35:29.208947 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:35:29.208952 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:35:29.209033 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:35:29.209128 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:35:29.210687 (+ 1559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:29.211444 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:29.212882 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:29.213400 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:29.213591 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:35:29.216187 (+ 2596us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:35:29.216192 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:35:29.216196 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:35:29.216233 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:35:29.219278 (+ 3045us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:35:29.220085 (+ 807us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:35:29.220090 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:35:29.220092 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:35:29.220238 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:35:29.220627 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:35:29.220630 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:35:29.220632 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:35:29.220965 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:35:29.221170 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0810 11:35:29.221563 (+ 393us) server_negotiation.cc:299] Negotiation successful
0810 11:35:29.221760 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260810 11:35:29.226974 25895 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60728:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 11:35:29.234681 25895 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 11:35:29.271132 26394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:29.251275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37842 (local address 127.8.18.193:45573)
0810 11:35:29.262802 (+ 11527us) server_negotiation.cc:206] Beginning negotiation
0810 11:35:29.262811 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 11:35:29.262846 (+ 35us) server_negotiation.cc:407] Connection header received
0810 11:35:29.262956 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:35:29.262964 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:35:29.263325 (+ 361us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:35:29.263628 (+ 303us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:35:29.264380 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:29.266085 (+ 1705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:29.268618 (+ 2533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:29.269799 (+ 1181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:29.269961 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:35:29.270233 (+ 272us) server_negotiation.cc:1035] Waiting for connection context
0810 11:35:29.270600 (+ 367us) server_negotiation.cc:299] Negotiation successful
0810 11:35:29.270781 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11330,"thread_start_us":11168,"threads_started":1}
I20260810 11:35:29.271214 26393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:29.250943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.193:45573 (local address 127.0.0.1:37842)
0810 11:35:29.251656 (+ 713us) negotiation.cc:107] Waiting for socket to connect
0810 11:35:29.251679 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0810 11:35:29.251821 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:35:29.263695 (+ 11874us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:35:29.263703 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:35:29.263741 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:35:29.264152 (+ 411us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:35:29.264179 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:35:29.266245 (+ 2066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:35:29.266250 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:35:29.268441 (+ 2191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:35:29.268457 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:35:29.270023 (+ 1566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:35:29.270039 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:35:29.270336 (+ 297us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:35:29.270380 (+ 44us) client_negotiation.cc:769] Sending connection context
0810 11:35:29.270536 (+ 156us) client_negotiation.cc:240] Negotiation successful
0810 11:35:29.270759 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":539,"mutex_wait_us":51,"thread_start_us":350,"threads_started":1}
I20260810 11:35:29.272862 26391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:29.249677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.195:40587 (local address 127.0.0.1:56052)
0810 11:35:29.250612 (+ 935us) negotiation.cc:107] Waiting for socket to connect
0810 11:35:29.250664 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0810 11:35:29.250886 (+ 222us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:35:29.265623 (+ 14737us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:35:29.265628 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:35:29.265656 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:35:29.266009 (+ 353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:35:29.266021 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:35:29.268256 (+ 2235us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:35:29.268262 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:35:29.269907 (+ 1645us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:35:29.269921 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:35:29.272103 (+ 2182us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:35:29.272109 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:35:29.272272 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:35:29.272304 (+ 32us) client_negotiation.cc:769] Sending connection context
0810 11:35:29.272411 (+ 107us) client_negotiation.cc:240] Negotiation successful
0810 11:35:29.272561 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":536,"thread_start_us":349,"threads_started":1}
I20260810 11:35:29.273073 26392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:29.249993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56052 (local address 127.8.18.195:40587)
0810 11:35:29.265013 (+ 15020us) server_negotiation.cc:206] Beginning negotiation
0810 11:35:29.265022 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 11:35:29.265055 (+ 33us) server_negotiation.cc:407] Connection header received
0810 11:35:29.265159 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:35:29.265166 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:35:29.265378 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:35:29.265617 (+ 239us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:35:29.266225 (+ 608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:29.268081 (+ 1856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:29.270081 (+ 2000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:29.271887 (+ 1806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:29.272037 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:35:29.272289 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0810 11:35:29.272478 (+ 189us) server_negotiation.cc:299] Negotiation successful
0810 11:35:29.272673 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14839,"thread_start_us":14655,"threads_started":1}
I20260810 11:35:29.275920 26029 tablet_service.cc:1467] Processing CreateTablet for tablet 35ae27ca4baa4217b70262bfc692eb6f (DEFAULT_TABLE table=test-table [id=c8ab604d9b834baa9c9b1401d95e2ed2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:35:29.276367 26311 tablet_service.cc:1467] Processing CreateTablet for tablet 35ae27ca4baa4217b70262bfc692eb6f (DEFAULT_TABLE table=test-table [id=c8ab604d9b834baa9c9b1401d95e2ed2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:35:29.277324 26029 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 35ae27ca4baa4217b70262bfc692eb6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:35:29.277688 26311 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 35ae27ca4baa4217b70262bfc692eb6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:35:29.283133 26396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:29.252320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57872 (local address 127.8.18.194:38411)
0810 11:35:29.263612 (+ 11292us) server_negotiation.cc:206] Beginning negotiation
0810 11:35:29.263620 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:35:29.263649 (+ 29us) server_negotiation.cc:407] Connection header received
0810 11:35:29.263753 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:35:29.263760 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:35:29.264000 (+ 240us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:35:29.264248 (+ 248us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:35:29.265026 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:29.266544 (+ 1518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:29.270886 (+ 4342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:29.272465 (+ 1579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:29.272629 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:35:29.272813 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0810 11:35:29.282467 (+ 9654us) server_negotiation.cc:299] Negotiation successful
0810 11:35:29.282729 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10756,"thread_start_us":10623,"threads_started":1}
I20260810 11:35:29.283951 26395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:29.251667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.194:38411 (local address 127.0.0.1:57872)
0810 11:35:29.261648 (+ 9981us) negotiation.cc:107] Waiting for socket to connect
0810 11:35:29.261675 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0810 11:35:29.261831 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:35:29.264281 (+ 2450us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:35:29.264288 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:35:29.264326 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:35:29.264812 (+ 486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:35:29.264824 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:35:29.266705 (+ 1881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:35:29.266711 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:35:29.270673 (+ 3962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:35:29.270687 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:35:29.281844 (+ 11157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:35:29.281854 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:35:29.282124 (+ 270us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:35:29.282174 (+ 50us) client_negotiation.cc:769] Sending connection context
0810 11:35:29.283064 (+ 890us) client_negotiation.cc:240] Negotiation successful
0810 11:35:29.283471 (+ 407us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9820,"mutex_wait_us":1,"thread_start_us":10272,"threads_started":1}
I20260810 11:35:29.288522 26167 tablet_service.cc:1467] Processing CreateTablet for tablet 35ae27ca4baa4217b70262bfc692eb6f (DEFAULT_TABLE table=test-table [id=c8ab604d9b834baa9c9b1401d95e2ed2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:35:29.290410 26167 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 35ae27ca4baa4217b70262bfc692eb6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:35:29.298905 26397 tablet_bootstrap.cc:492] T 35ae27ca4baa4217b70262bfc692eb6f P 39130be9337646e0b2b1b0f5e025e1c7: Bootstrap starting.
I20260810 11:35:29.303923 26397 tablet_bootstrap.cc:654] T 35ae27ca4baa4217b70262bfc692eb6f P 39130be9337646e0b2b1b0f5e025e1c7: Neither blocks nor log segments found. Creating new log.
I20260810 11:35:29.305395 26397 log.cc:824] T 35ae27ca4baa4217b70262bfc692eb6f P 39130be9337646e0b2b1b0f5e025e1c7: Log is configured to *not* fsync() on all Append() calls
I20260810 11:35:29.306262 26398 tablet_bootstrap.cc:492] T 35ae27ca4baa4217b70262bfc692eb6f P 0ff2be4158114536973e89ffaf108d7a: Bootstrap starting.
I20260810 11:35:29.307605 26400 tablet_bootstrap.cc:492] T 35ae27ca4baa4217b70262bfc692eb6f P 62fc62a2e30244009741c41455bfe3c9: Bootstrap starting.
I20260810 11:35:29.310551 26397 tablet_bootstrap.cc:492] T 35ae27ca4baa4217b70262bfc692eb6f P 39130be9337646e0b2b1b0f5e025e1c7: No bootstrap required, opened a new log
I20260810 11:35:29.310786 26398 tablet_bootstrap.cc:654] T 35ae27ca4baa4217b70262bfc692eb6f P 0ff2be4158114536973e89ffaf108d7a: Neither blocks nor log segments found. Creating new log.
I20260810 11:35:29.310881 26397 ts_tablet_manager.cc:1397] T 35ae27ca4baa4217b70262bfc692eb6f P 39130be9337646e0b2b1b0f5e025e1c7: Time spent bootstrapping tablet: real 0.012s user 0.004s sys 0.003s
I20260810 11:35:29.311738 26400 tablet_bootstrap.cc:654] T 35ae27ca4baa4217b70262bfc692eb6f P 62fc62a2e30244009741c41455bfe3c9: Neither blocks nor log segments found. Creating new log.
I20260810 11:35:29.312059 26398 log.cc:824] T 35ae27ca4baa4217b70262bfc692eb6f P 0ff2be4158114536973e89ffaf108d7a: Log is configured to *not* fsync() on all Append() calls
I20260810 11:35:29.313123 26400 log.cc:824] T 35ae27ca4baa4217b70262bfc692eb6f P 62fc62a2e30244009741c41455bfe3c9: Log is configured to *not* fsync() on all Append() calls
I20260810 11:35:29.321519 26400 tablet_bootstrap.cc:492] T 35ae27ca4baa4217b70262bfc692eb6f P 62fc62a2e30244009741c41455bfe3c9: No bootstrap required, opened a new log
I20260810 11:35:29.321677 26398 tablet_bootstrap.cc:492] T 35ae27ca4baa4217b70262bfc692eb6f P 0ff2be4158114536973e89ffaf108d7a: No bootstrap required, opened a new log
I20260810 11:35:29.321918 26400 ts_tablet_manager.cc:1397] T 35ae27ca4baa4217b70262bfc692eb6f P 62fc62a2e30244009741c41455bfe3c9: Time spent bootstrapping tablet: real 0.015s user 0.008s sys 0.003s
I20260810 11:35:29.321990 26398 ts_tablet_manager.cc:1397] T 35ae27ca4baa4217b70262bfc692eb6f P 0ff2be4158114536973e89ffaf108d7a: Time spent bootstrapping tablet: real 0.016s user 0.003s sys 0.011s
I20260810 11:35:29.323189 26397 raft_consensus.cc:348] T 35ae27ca4baa4217b70262bfc692eb6f P 39130be9337646e0b2b1b0f5e025e1c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39130be9337646e0b2b1b0f5e025e1c7" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 40587 } } peers { permanent_uuid: "0ff2be4158114536973e89ffaf108d7a" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 45573 } } peers { permanent_uuid: "62fc62a2e30244009741c41455bfe3c9" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 38411 } }
I20260810 11:35:29.323721 26397 raft_consensus.cc:374] T 35ae27ca4baa4217b70262bfc692eb6f P 39130be9337646e0b2b1b0f5e025e1c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:35:29.323827 26397 raft_consensus.cc:729] T 35ae27ca4baa4217b70262bfc692eb6f P 39130be9337646e0b2b1b0f5e025e1c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39130be9337646e0b2b1b0f5e025e1c7, State: Initialized, Role: FOLLOWER
I20260810 11:35:29.324714 26397 consensus_queue.cc:260] T 35ae27ca4baa4217b70262bfc692eb6f P 39130be9337646e0b2b1b0f5e025e1c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39130be9337646e0b2b1b0f5e025e1c7" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 40587 } } peers { permanent_uuid: "0ff2be4158114536973e89ffaf108d7a" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 45573 } } peers { permanent_uuid: "62fc62a2e30244009741c41455bfe3c9" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 38411 } }
I20260810 11:35:29.326115 26377 heartbeater.cc:503] Master 127.8.18.254:38989 was elected leader, sending a full tablet report...
I20260810 11:35:29.326886 26397 ts_tablet_manager.cc:1428] T 35ae27ca4baa4217b70262bfc692eb6f P 39130be9337646e0b2b1b0f5e025e1c7: Time spent starting tablet: real 0.016s user 0.014s sys 0.002s
I20260810 11:35:29.331796 26400 raft_consensus.cc:348] T 35ae27ca4baa4217b70262bfc692eb6f P 62fc62a2e30244009741c41455bfe3c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39130be9337646e0b2b1b0f5e025e1c7" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 40587 } } peers { permanent_uuid: "0ff2be4158114536973e89ffaf108d7a" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 45573 } } peers { permanent_uuid: "62fc62a2e30244009741c41455bfe3c9" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 38411 } }
I20260810 11:35:29.332360 26400 raft_consensus.cc:374] T 35ae27ca4baa4217b70262bfc692eb6f P 62fc62a2e30244009741c41455bfe3c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:35:29.332486 26400 raft_consensus.cc:729] T 35ae27ca4baa4217b70262bfc692eb6f P 62fc62a2e30244009741c41455bfe3c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62fc62a2e30244009741c41455bfe3c9, State: Initialized, Role: FOLLOWER
I20260810 11:35:29.333353 26400 consensus_queue.cc:260] T 35ae27ca4baa4217b70262bfc692eb6f P 62fc62a2e30244009741c41455bfe3c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39130be9337646e0b2b1b0f5e025e1c7" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 40587 } } peers { permanent_uuid: "0ff2be4158114536973e89ffaf108d7a" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 45573 } } peers { permanent_uuid: "62fc62a2e30244009741c41455bfe3c9" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 38411 } }
I20260810 11:35:29.333317 26398 raft_consensus.cc:348] T 35ae27ca4baa4217b70262bfc692eb6f P 0ff2be4158114536973e89ffaf108d7a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39130be9337646e0b2b1b0f5e025e1c7" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 40587 } } peers { permanent_uuid: "0ff2be4158114536973e89ffaf108d7a" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 45573 } } peers { permanent_uuid: "62fc62a2e30244009741c41455bfe3c9" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 38411 } }
I20260810 11:35:29.333801 26398 raft_consensus.cc:374] T 35ae27ca4baa4217b70262bfc692eb6f P 0ff2be4158114536973e89ffaf108d7a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:35:29.333930 26398 raft_consensus.cc:729] T 35ae27ca4baa4217b70262bfc692eb6f P 0ff2be4158114536973e89ffaf108d7a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ff2be4158114536973e89ffaf108d7a, State: Initialized, Role: FOLLOWER
I20260810 11:35:29.334650 26398 consensus_queue.cc:260] T 35ae27ca4baa4217b70262bfc692eb6f P 0ff2be4158114536973e89ffaf108d7a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39130be9337646e0b2b1b0f5e025e1c7" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 40587 } } peers { permanent_uuid: "0ff2be4158114536973e89ffaf108d7a" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 45573 } } peers { permanent_uuid: "62fc62a2e30244009741c41455bfe3c9" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 38411 } }
I20260810 11:35:29.334790 26234 heartbeater.cc:503] Master 127.8.18.254:38989 was elected leader, sending a full tablet report...
I20260810 11:35:29.335544 26400 ts_tablet_manager.cc:1428] T 35ae27ca4baa4217b70262bfc692eb6f P 62fc62a2e30244009741c41455bfe3c9: Time spent starting tablet: real 0.013s user 0.005s sys 0.007s
I20260810 11:35:29.336278 26398 ts_tablet_manager.cc:1428] T 35ae27ca4baa4217b70262bfc692eb6f P 0ff2be4158114536973e89ffaf108d7a: Time spent starting tablet: real 0.014s user 0.012s sys 0.001s
W20260810 11:35:29.341318 26235 tablet.cc:2369] T 35ae27ca4baa4217b70262bfc692eb6f P 62fc62a2e30244009741c41455bfe3c9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 11:35:29.361935 26404 raft_consensus.cc:482] T 35ae27ca4baa4217b70262bfc692eb6f P 39130be9337646e0b2b1b0f5e025e1c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 11:35:29.362376 26404 raft_consensus.cc:504] T 35ae27ca4baa4217b70262bfc692eb6f P 39130be9337646e0b2b1b0f5e025e1c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39130be9337646e0b2b1b0f5e025e1c7" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 40587 } } peers { permanent_uuid: "0ff2be4158114536973e89ffaf108d7a" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 45573 } } peers { permanent_uuid: "62fc62a2e30244009741c41455bfe3c9" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 38411 } }
I20260810 11:35:29.364354 26404 leader_election.cc:288] T 35ae27ca4baa4217b70262bfc692eb6f P 39130be9337646e0b2b1b0f5e025e1c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ff2be4158114536973e89ffaf108d7a (127.8.18.193:45573), 62fc62a2e30244009741c41455bfe3c9 (127.8.18.194:38411)
I20260810 11:35:29.371812 26394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:29.364905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.195:33665 (local address 127.8.18.193:45573)
0810 11:35:29.365154 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0810 11:35:29.365161 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:35:29.365184 (+ 23us) server_negotiation.cc:407] Connection header received
0810 11:35:29.365271 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:35:29.365279 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:35:29.365356 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:35:29.365538 (+ 182us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:35:29.366248 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:29.367307 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:29.369348 (+ 2041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:29.370725 (+ 1377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:29.370858 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:35:29.370916 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0810 11:35:29.371459 (+ 543us) server_negotiation.cc:299] Negotiation successful
0810 11:35:29.371597 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260810 11:35:29.371977 26380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:29.364713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.193:45573 (local address 127.8.18.195:33665)
0810 11:35:29.364940 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0810 11:35:29.364963 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0810 11:35:29.365116 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:35:29.365803 (+ 687us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:35:29.365808 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:35:29.365850 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:35:29.366042 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:35:29.366052 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:35:29.367443 (+ 1391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:35:29.367449 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:35:29.369150 (+ 1701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:35:29.369163 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:35:29.370921 (+ 1758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:35:29.370930 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:35:29.371232 (+ 302us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:35:29.371273 (+ 41us) client_negotiation.cc:769] Sending connection context
0810 11:35:29.371416 (+ 143us) client_negotiation.cc:240] Negotiation successful
0810 11:35:29.371596 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":62,"mutex_wait_us":23}
I20260810 11:35:29.373195 26407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:29.365202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.194:38411 (local address 127.8.18.195:48227)
0810 11:35:29.365938 (+ 736us) negotiation.cc:107] Waiting for socket to connect
0810 11:35:29.365954 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 11:35:29.366065 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:35:29.366399 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:35:29.366404 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:35:29.366431 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:35:29.366842 (+ 411us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:35:29.366856 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:35:29.368104 (+ 1248us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:35:29.368108 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:35:29.370362 (+ 2254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:35:29.370383 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:35:29.372328 (+ 1945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:35:29.372334 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:35:29.372534 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:35:29.372615 (+ 81us) client_negotiation.cc:769] Sending connection context
0810 11:35:29.372734 (+ 119us) client_negotiation.cc:240] Negotiation successful
0810 11:35:29.372881 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":627,"mutex_wait_us":5,"thread_start_us":401,"threads_started":1}
I20260810 11:35:29.373344 26396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:29.365457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.195:48227 (local address 127.8.18.194:38411)
0810 11:35:29.365709 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0810 11:35:29.365714 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:35:29.366054 (+ 340us) server_negotiation.cc:407] Connection header received
0810 11:35:29.366202 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:35:29.366208 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:35:29.366284 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:35:29.366353 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:35:29.367005 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:29.367942 (+ 937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:29.370626 (+ 2684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:29.372038 (+ 1412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:29.372242 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:35:29.372338 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0810 11:35:29.372848 (+ 510us) server_negotiation.cc:299] Negotiation successful
0810 11:35:29.373042 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260810 11:35:29.373474 26050 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "35ae27ca4baa4217b70262bfc692eb6f" candidate_uuid: "39130be9337646e0b2b1b0f5e025e1c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ff2be4158114536973e89ffaf108d7a" is_pre_election: true
I20260810 11:35:29.374249 26050 raft_consensus.cc:2454] T 35ae27ca4baa4217b70262bfc692eb6f P 0ff2be4158114536973e89ffaf108d7a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39130be9337646e0b2b1b0f5e025e1c7 in term 0.
I20260810 11:35:29.374554 26188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "35ae27ca4baa4217b70262bfc692eb6f" candidate_uuid: "39130be9337646e0b2b1b0f5e025e1c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62fc62a2e30244009741c41455bfe3c9" is_pre_election: true
I20260810 11:35:29.375137 26258 leader_election.cc:302] T 35ae27ca4baa4217b70262bfc692eb6f P 39130be9337646e0b2b1b0f5e025e1c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ff2be4158114536973e89ffaf108d7a, 39130be9337646e0b2b1b0f5e025e1c7; no voters:
I20260810 11:35:29.375190 26188 raft_consensus.cc:2454] T 35ae27ca4baa4217b70262bfc692eb6f P 62fc62a2e30244009741c41455bfe3c9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39130be9337646e0b2b1b0f5e025e1c7 in term 0.
I20260810 11:35:29.375699 26404 raft_consensus.cc:2781] T 35ae27ca4baa4217b70262bfc692eb6f P 39130be9337646e0b2b1b0f5e025e1c7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 11:35:29.375916 26404 raft_consensus.cc:482] T 35ae27ca4baa4217b70262bfc692eb6f P 39130be9337646e0b2b1b0f5e025e1c7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 11:35:29.376083 26404 raft_consensus.cc:3037] T 35ae27ca4baa4217b70262bfc692eb6f P 39130be9337646e0b2b1b0f5e025e1c7 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:35:29.378821 26404 raft_consensus.cc:504] T 35ae27ca4baa4217b70262bfc692eb6f P 39130be9337646e0b2b1b0f5e025e1c7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39130be9337646e0b2b1b0f5e025e1c7" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 40587 } } peers { permanent_uuid: "0ff2be4158114536973e89ffaf108d7a" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 45573 } } peers { permanent_uuid: "62fc62a2e30244009741c41455bfe3c9" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 38411 } }
I20260810 11:35:29.379487 26404 leader_election.cc:288] T 35ae27ca4baa4217b70262bfc692eb6f P 39130be9337646e0b2b1b0f5e025e1c7 [CANDIDATE]: Term 1 election: Requested vote from peers 0ff2be4158114536973e89ffaf108d7a (127.8.18.193:45573), 62fc62a2e30244009741c41455bfe3c9 (127.8.18.194:38411)
I20260810 11:35:29.380208 26050 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "35ae27ca4baa4217b70262bfc692eb6f" candidate_uuid: "39130be9337646e0b2b1b0f5e025e1c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ff2be4158114536973e89ffaf108d7a"
I20260810 11:35:29.380295 26188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "35ae27ca4baa4217b70262bfc692eb6f" candidate_uuid: "39130be9337646e0b2b1b0f5e025e1c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62fc62a2e30244009741c41455bfe3c9"
I20260810 11:35:29.380451 26050 raft_consensus.cc:3037] T 35ae27ca4baa4217b70262bfc692eb6f P 0ff2be4158114536973e89ffaf108d7a [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:35:29.380622 26188 raft_consensus.cc:3037] T 35ae27ca4baa4217b70262bfc692eb6f P 62fc62a2e30244009741c41455bfe3c9 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:35:29.384280 26050 raft_consensus.cc:2454] T 35ae27ca4baa4217b70262bfc692eb6f P 0ff2be4158114536973e89ffaf108d7a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39130be9337646e0b2b1b0f5e025e1c7 in term 1.
I20260810 11:35:29.384711 26188 raft_consensus.cc:2454] T 35ae27ca4baa4217b70262bfc692eb6f P 62fc62a2e30244009741c41455bfe3c9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39130be9337646e0b2b1b0f5e025e1c7 in term 1.
I20260810 11:35:29.384905 26258 leader_election.cc:302] T 35ae27ca4baa4217b70262bfc692eb6f P 39130be9337646e0b2b1b0f5e025e1c7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ff2be4158114536973e89ffaf108d7a, 39130be9337646e0b2b1b0f5e025e1c7; no voters:
I20260810 11:35:29.385371 26404 raft_consensus.cc:2781] T 35ae27ca4baa4217b70262bfc692eb6f P 39130be9337646e0b2b1b0f5e025e1c7 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:35:29.386026 26404 raft_consensus.cc:686] T 35ae27ca4baa4217b70262bfc692eb6f P 39130be9337646e0b2b1b0f5e025e1c7 [term 1 LEADER]: Becoming Leader. State: Replica: 39130be9337646e0b2b1b0f5e025e1c7, State: Running, Role: LEADER
I20260810 11:35:29.386615 26404 consensus_queue.cc:237] T 35ae27ca4baa4217b70262bfc692eb6f P 39130be9337646e0b2b1b0f5e025e1c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39130be9337646e0b2b1b0f5e025e1c7" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 40587 } } peers { permanent_uuid: "0ff2be4158114536973e89ffaf108d7a" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 45573 } } peers { permanent_uuid: "62fc62a2e30244009741c41455bfe3c9" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 38411 } }
I20260810 11:35:29.392735 25894 catalog_manager.cc:5495] T 35ae27ca4baa4217b70262bfc692eb6f P 39130be9337646e0b2b1b0f5e025e1c7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 39130be9337646e0b2b1b0f5e025e1c7 (127.8.18.195). New cstate: current_term: 1 leader_uuid: "39130be9337646e0b2b1b0f5e025e1c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39130be9337646e0b2b1b0f5e025e1c7" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 40587 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0ff2be4158114536973e89ffaf108d7a" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 45573 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "62fc62a2e30244009741c41455bfe3c9" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 38411 } health_report { overall_health: UNKNOWN } } }
I20260810 11:35:29.421303 26392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:29.415360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56068 (local address 127.8.18.195:40587)
0810 11:35:29.415573 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0810 11:35:29.415581 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:35:29.415617 (+ 36us) server_negotiation.cc:407] Connection header received
0810 11:35:29.415889 (+ 272us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:35:29.415893 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:35:29.415986 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:35:29.416188 (+ 202us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 11:35:29.416594 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:29.417332 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:29.418845 (+ 1513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:29.419309 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:29.419457 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:35:29.420055 (+ 598us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 11:35:29.420629 (+ 574us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 11:35:29.420801 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0810 11:35:29.420909 (+ 108us) server_negotiation.cc:299] Negotiation successful
0810 11:35:29.421005 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
W20260810 11:35:29.421679 26378 tablet.cc:2369] T 35ae27ca4baa4217b70262bfc692eb6f P 39130be9337646e0b2b1b0f5e025e1c7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 11:35:29.431571 26188 raft_consensus.cc:1261] T 35ae27ca4baa4217b70262bfc692eb6f P 62fc62a2e30244009741c41455bfe3c9 [term 1 FOLLOWER]: Refusing update from remote peer 39130be9337646e0b2b1b0f5e025e1c7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 11:35:29.431561 26050 raft_consensus.cc:1261] T 35ae27ca4baa4217b70262bfc692eb6f P 0ff2be4158114536973e89ffaf108d7a [term 1 FOLLOWER]: Refusing update from remote peer 39130be9337646e0b2b1b0f5e025e1c7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 11:35:29.432765 26408 consensus_queue.cc:1035] T 35ae27ca4baa4217b70262bfc692eb6f P 39130be9337646e0b2b1b0f5e025e1c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "62fc62a2e30244009741c41455bfe3c9" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 38411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 11:35:29.433373 26404 consensus_queue.cc:1035] T 35ae27ca4baa4217b70262bfc692eb6f P 39130be9337646e0b2b1b0f5e025e1c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ff2be4158114536973e89ffaf108d7a" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 45573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 11:35:29.459553 25895 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60728:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 11:35:29.459972 25895 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60728:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 11:35:29.465699 25895 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 73ec08971ce84042909f6b92427ef790: Sending DeleteTablet for 3 replicas of tablet 35ae27ca4baa4217b70262bfc692eb6f
I20260810 11:35:29.467231 26029 tablet_service.cc:1514] Processing DeleteTablet for tablet 35ae27ca4baa4217b70262bfc692eb6f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:35:29 UTC) from {username='kudu', principal='kudu/127.8.18.254@KRBTEST.COM'} at 127.0.0.1:37842
I20260810 11:35:29.467648 26028 tablet_service.cc:1514] Processing DeleteTablet for tablet 35ae27ca4baa4217b70262bfc692eb6f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:35:29 UTC) from {username='kudu', principal='kudu/127.8.18.254@KRBTEST.COM'} at 127.0.0.1:37842
I20260810 11:35:29.468214 26167 tablet_service.cc:1514] Processing DeleteTablet for tablet 35ae27ca4baa4217b70262bfc692eb6f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:35:29 UTC) from {username='kudu', principal='kudu/127.8.18.254@KRBTEST.COM'} at 127.0.0.1:57872
I20260810 11:35:29.469147 26311 tablet_service.cc:1514] Processing DeleteTablet for tablet 35ae27ca4baa4217b70262bfc692eb6f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:35:29 UTC) from {username='kudu', principal='kudu/127.8.18.254@KRBTEST.COM'} at 127.0.0.1:56052
I20260810 11:35:29.469373 26422 tablet_replica.cc:330] T 35ae27ca4baa4217b70262bfc692eb6f P 62fc62a2e30244009741c41455bfe3c9: stopping tablet replica
I20260810 11:35:29.470075 26422 raft_consensus.cc:2229] T 35ae27ca4baa4217b70262bfc692eb6f P 62fc62a2e30244009741c41455bfe3c9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 11:35:29.470655 26422 raft_consensus.cc:2258] T 35ae27ca4baa4217b70262bfc692eb6f P 62fc62a2e30244009741c41455bfe3c9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:35:29.471343 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 25957
I20260810 11:35:29.477700 26423 tablet_replica.cc:330] T 35ae27ca4baa4217b70262bfc692eb6f P 39130be9337646e0b2b1b0f5e025e1c7: stopping tablet replica
I20260810 11:35:29.477975 26421 tablet_replica.cc:330] T 35ae27ca4baa4217b70262bfc692eb6f P 0ff2be4158114536973e89ffaf108d7a: stopping tablet replica
I20260810 11:35:29.478246 26423 raft_consensus.cc:2229] T 35ae27ca4baa4217b70262bfc692eb6f P 39130be9337646e0b2b1b0f5e025e1c7 [term 1 LEADER]: Raft consensus shutting down.
I20260810 11:35:29.478547 26421 raft_consensus.cc:2229] T 35ae27ca4baa4217b70262bfc692eb6f P 0ff2be4158114536973e89ffaf108d7a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 11:35:29.478753 26422 ts_tablet_manager.cc:1905] T 35ae27ca4baa4217b70262bfc692eb6f P 62fc62a2e30244009741c41455bfe3c9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:35:29.479056 26423 raft_consensus.cc:2258] T 35ae27ca4baa4217b70262bfc692eb6f P 39130be9337646e0b2b1b0f5e025e1c7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:35:29.479286 26421 raft_consensus.cc:2258] T 35ae27ca4baa4217b70262bfc692eb6f P 0ff2be4158114536973e89ffaf108d7a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:35:29.481484 26421 ts_tablet_manager.cc:1905] T 35ae27ca4baa4217b70262bfc692eb6f P 0ff2be4158114536973e89ffaf108d7a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:35:29.481495 26423 ts_tablet_manager.cc:1905] T 35ae27ca4baa4217b70262bfc692eb6f P 39130be9337646e0b2b1b0f5e025e1c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:35:29.486481 26422 ts_tablet_manager.cc:1918] T 35ae27ca4baa4217b70262bfc692eb6f P 62fc62a2e30244009741c41455bfe3c9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 11:35:29.486711 26422 log.cc:1196] T 35ae27ca4baa4217b70262bfc692eb6f P 62fc62a2e30244009741c41455bfe3c9: Deleting WAL directory at /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786361401922362-8267-0/minicluster-data/ts-1/wal/wals/35ae27ca4baa4217b70262bfc692eb6f
I20260810 11:35:29.487347 26422 ts_tablet_manager.cc:1939] T 35ae27ca4baa4217b70262bfc692eb6f P 62fc62a2e30244009741c41455bfe3c9: Deleting consensus metadata
I20260810 11:35:29.489574 26421 ts_tablet_manager.cc:1918] T 35ae27ca4baa4217b70262bfc692eb6f P 0ff2be4158114536973e89ffaf108d7a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 11:35:29.489645 25879 catalog_manager.cc:4841] TS 62fc62a2e30244009741c41455bfe3c9 (127.8.18.194:38411): tablet 35ae27ca4baa4217b70262bfc692eb6f (table test-table [id=c8ab604d9b834baa9c9b1401d95e2ed2]) successfully deleted
I20260810 11:35:29.489895 26423 ts_tablet_manager.cc:1918] T 35ae27ca4baa4217b70262bfc692eb6f P 39130be9337646e0b2b1b0f5e025e1c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 11:35:29.489966 26421 log.cc:1196] T 35ae27ca4baa4217b70262bfc692eb6f P 0ff2be4158114536973e89ffaf108d7a: Deleting WAL directory at /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786361401922362-8267-0/minicluster-data/ts-0/wal/wals/35ae27ca4baa4217b70262bfc692eb6f
I20260810 11:35:29.490082 26423 log.cc:1196] T 35ae27ca4baa4217b70262bfc692eb6f P 39130be9337646e0b2b1b0f5e025e1c7: Deleting WAL directory at /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786361401922362-8267-0/minicluster-data/ts-2/wal/wals/35ae27ca4baa4217b70262bfc692eb6f
I20260810 11:35:29.490706 26421 ts_tablet_manager.cc:1939] T 35ae27ca4baa4217b70262bfc692eb6f P 0ff2be4158114536973e89ffaf108d7a: Deleting consensus metadata
I20260810 11:35:29.490718 26423 ts_tablet_manager.cc:1939] T 35ae27ca4baa4217b70262bfc692eb6f P 39130be9337646e0b2b1b0f5e025e1c7: Deleting consensus metadata
I20260810 11:35:29.492681 25879 catalog_manager.cc:4841] TS 39130be9337646e0b2b1b0f5e025e1c7 (127.8.18.195:40587): tablet 35ae27ca4baa4217b70262bfc692eb6f (table test-table [id=c8ab604d9b834baa9c9b1401d95e2ed2]) successfully deleted
I20260810 11:35:29.493021 25879 catalog_manager.cc:4841] TS 0ff2be4158114536973e89ffaf108d7a (127.8.18.193:45573): tablet 35ae27ca4baa4217b70262bfc692eb6f (table test-table [id=c8ab604d9b834baa9c9b1401d95e2ed2]) successfully deleted
W20260810 11:35:29.493505 25879 catalog_manager.cc:4808] TS 0ff2be4158114536973e89ffaf108d7a (127.8.18.193:45573): delete failed for tablet 35ae27ca4baa4217b70262bfc692eb6f because the tablet was not found. No further retry: Not found: Tablet not found: 35ae27ca4baa4217b70262bfc692eb6f
Aug 10 11:35:29 dist-test-slave-srgv krb5kdc[25849](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361726, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.18.193@KRBTEST.COM
I20260810 11:35:29.508177 26394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:29.475858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37846 (local address 127.8.18.193:45573)
0810 11:35:29.476125 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0810 11:35:29.476143 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0810 11:35:29.477453 (+ 1310us) server_negotiation.cc:407] Connection header received
0810 11:35:29.489846 (+ 12393us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:35:29.489852 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:35:29.489943 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:35:29.490056 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:35:29.492184 (+ 2128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:29.493431 (+ 1247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:29.495576 (+ 2145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:29.496003 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:29.496111 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:35:29.501577 (+ 5466us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:35:29.501596 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:35:29.501611 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:35:29.501659 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:35:29.504789 (+ 3130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:35:29.505803 (+ 1014us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:35:29.505811 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:35:29.505814 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:35:29.505964 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:35:29.506415 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:35:29.506419 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:35:29.506421 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:35:29.506825 (+ 404us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:35:29.507190 (+ 365us) server_negotiation.cc:1035] Waiting for connection context
0810 11:35:29.507516 (+ 326us) server_negotiation.cc:299] Negotiation successful
0810 11:35:29.507766 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260810 11:35:29.508955 26091 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:35:29.623219 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 25957
I20260810 11:35:29.648027 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 26104
Aug 10 11:35:29 dist-test-slave-srgv krb5kdc[25849](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361726, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.18.194@KRBTEST.COM
I20260810 11:35:29.667572 26396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:29.648935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57888 (local address 127.8.18.194:38411)
0810 11:35:29.649144 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0810 11:35:29.649151 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:35:29.649238 (+ 87us) server_negotiation.cc:407] Connection header received
0810 11:35:29.649609 (+ 371us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:35:29.649616 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:35:29.649721 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:35:29.649842 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:35:29.651350 (+ 1508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:29.652481 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:29.653987 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:29.654596 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:29.654731 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:35:29.659647 (+ 4916us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:35:29.659673 (+ 26us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:35:29.659695 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:35:29.659758 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:35:29.663995 (+ 4237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:35:29.664928 (+ 933us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:35:29.664939 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:35:29.664942 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:35:29.665121 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:35:29.665699 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:35:29.665707 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:35:29.665711 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:35:29.666277 (+ 566us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:35:29.666606 (+ 329us) server_negotiation.cc:1035] Waiting for connection context
0810 11:35:29.666921 (+ 315us) server_negotiation.cc:299] Negotiation successful
0810 11:35:29.667176 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260810 11:35:29.668486 26228 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:35:29.765211 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 26104
I20260810 11:35:29.793978 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 26240
Aug 10 11:35:29 dist-test-slave-srgv krb5kdc[25849](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361726, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.18.195@KRBTEST.COM
I20260810 11:35:29.812723 26392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:29.795044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56082 (local address 127.8.18.195:40587)
0810 11:35:29.795332 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0810 11:35:29.795338 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:35:29.795355 (+ 17us) server_negotiation.cc:407] Connection header received
0810 11:35:29.795411 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:35:29.795415 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:35:29.795486 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:35:29.795602 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:35:29.797056 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:29.798100 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:29.799565 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:29.800051 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:29.800222 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:35:29.805629 (+ 5407us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:35:29.805648 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:35:29.805668 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:35:29.805713 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:35:29.809260 (+ 3547us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:35:29.810234 (+ 974us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:35:29.810242 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:35:29.810244 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:35:29.810401 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:35:29.810864 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:35:29.810877 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:35:29.810880 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:35:29.811313 (+ 433us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:35:29.811716 (+ 403us) server_negotiation.cc:1035] Waiting for connection context
0810 11:35:29.812050 (+ 334us) server_negotiation.cc:299] Negotiation successful
0810 11:35:29.812319 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124}
I20260810 11:35:29.813777 26372 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:35:29.948709 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 26240
I20260810 11:35:29.973258 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 25862
I20260810 11:35:29.974690 25926 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:35:30.091114 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 25862
2026-08-10T11:35:30Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (4783 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-10T11:35:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T11:35:30Z Disabled control of system clock
I20260810 11:35:30.191617 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786361401922362-8267-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786361401922362-8267-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786361401922362-8267-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.18.254:40883
--webserver_interface=127.8.18.254
--webserver_port=0
--builtin_ntp_servers=127.8.18.212:35867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.18.254:40883
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 11:35:30.564074 26432 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:35:30.564404 26432 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:35:30.564559 26432 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:35:30.572610 26432 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 11:35:30.572738 26432 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:35:30.572805 26432 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:35:30.572891 26432 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 11:35:30.572970 26432 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 11:35:30.583103 26432 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.18.212:35867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786361401922362-8267-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786361401922362-8267-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.18.254:40883
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.18.254:40883
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.18.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786361401922362-8267-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:35:30.584566 26432 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:35:30.586973 26432 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:35:30.602732 26438 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:35:30.602747 26437 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:35:30.602785 26440 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:35:30.603739 26432 server_base.cc:1034] running on GCE node
I20260810 11:35:30.604708 26432 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:35:30.606482 26432 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:35:30.607764 26432 hybrid_clock.cc:648] HybridClock initialized: now 1786361730607731 us; error 80 us; skew 500 ppm
I20260810 11:35:30.608227 26432 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 11:35:30.616166 26432 webserver.cc:460] Webserver started at http://127.8.18.254:45737/ using document root <none> and password file <none>
I20260810 11:35:30.617056 26432 fs_manager.cc:359] Metadata directory not provided
I20260810 11:35:30.617224 26432 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:35:30.617683 26432 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:35:30.620973 26432 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786361401922362-8267-0/minicluster-data/master-0/data/instance:
uuid: "86a17c33afae4bef9c8d76a42011e6e2"
format_stamp: "Formatted at 2026-08-10 11:35:30 on dist-test-slave-srgv"
server_key: "f074638a37fc76734c895842d1c0e5a9"
server_key_iv: "f7476c1e8d3d9fb3465a2f8000ae32a7"
server_key_version: "encryptionkey@0"
I20260810 11:35:30.621973 26432 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786361401922362-8267-0/minicluster-data/master-0/wal/instance:
uuid: "86a17c33afae4bef9c8d76a42011e6e2"
format_stamp: "Formatted at 2026-08-10 11:35:30 on dist-test-slave-srgv"
server_key: "f074638a37fc76734c895842d1c0e5a9"
server_key_iv: "f7476c1e8d3d9fb3465a2f8000ae32a7"
server_key_version: "encryptionkey@0"
I20260810 11:35:30.628057 26432 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.001s
I20260810 11:35:30.632472 26447 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:35:30.634387 26432 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260810 11:35:30.634656 26432 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786361401922362-8267-0/minicluster-data/master-0/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786361401922362-8267-0/minicluster-data/master-0/wal
uuid: "86a17c33afae4bef9c8d76a42011e6e2"
format_stamp: "Formatted at 2026-08-10 11:35:30 on dist-test-slave-srgv"
server_key: "f074638a37fc76734c895842d1c0e5a9"
server_key_iv: "f7476c1e8d3d9fb3465a2f8000ae32a7"
server_key_version: "encryptionkey@0"
I20260810 11:35:30.634927 26432 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786361401922362-8267-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786361401922362-8267-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786361401922362-8267-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:35:30.667389 26432 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:35:30.668372 26432 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:35:30.668717 26432 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:35:30.690356 26432 rpc_server.cc:307] RPC server started. Bound to: 127.8.18.254:40883
I20260810 11:35:30.690394 26498 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.18.254:40883 every 8 connection(s)
I20260810 11:35:30.691990 26432 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
I20260810 11:35:30.695221 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 26432
I20260810 11:35:30.695439 8267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786361401922362-8267-0/minicluster-data/master-0/wal/instance
I20260810 11:35:30.695935 8267 external_mini_cluster.cc:1428] Setting key da5e49a01dd65c5966a37268fbeacf83
I20260810 11:35:30.696720 26499 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:35:30.713836 26499 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 86a17c33afae4bef9c8d76a42011e6e2: Bootstrap starting.
I20260810 11:35:30.714406 26502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:30.698200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59070 (local address 127.8.18.254:40883)
0810 11:35:30.699420 (+ 1220us) server_negotiation.cc:206] Beginning negotiation
0810 11:35:30.699436 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0810 11:35:30.699496 (+ 60us) server_negotiation.cc:407] Connection header received
0810 11:35:30.702201 (+ 2705us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:35:30.702242 (+ 41us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:35:30.703031 (+ 789us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:35:30.703582 (+ 551us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:35:30.704283 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:30.706487 (+ 2204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:30.708202 (+ 1715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:30.709442 (+ 1240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:30.709699 (+ 257us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:35:30.711112 (+ 1413us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:35:30.711123 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:35:30.711162 (+ 39us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:35:30.711968 (+ 806us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:35:30.712085 (+ 117us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:35:30.712125 (+ 40us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:35:30.712268 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0810 11:35:30.712533 (+ 265us) server_negotiation.cc:299] Negotiation successful
0810 11:35:30.712925 (+ 392us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":636,"thread_start_us":394,"threads_started":1}
I20260810 11:35:30.717607 26499 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 86a17c33afae4bef9c8d76a42011e6e2: Neither blocks nor log segments found. Creating new log.
I20260810 11:35:30.718768 26499 log.cc:824] T 00000000000000000000000000000000 P 86a17c33afae4bef9c8d76a42011e6e2: Log is configured to *not* fsync() on all Append() calls
I20260810 11:35:30.722079 26499 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 86a17c33afae4bef9c8d76a42011e6e2: No bootstrap required, opened a new log
I20260810 11:35:30.730902 26499 raft_consensus.cc:348] T 00000000000000000000000000000000 P 86a17c33afae4bef9c8d76a42011e6e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86a17c33afae4bef9c8d76a42011e6e2" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 40883 } }
I20260810 11:35:30.731330 26499 raft_consensus.cc:374] T 00000000000000000000000000000000 P 86a17c33afae4bef9c8d76a42011e6e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:35:30.731413 26499 raft_consensus.cc:729] T 00000000000000000000000000000000 P 86a17c33afae4bef9c8d76a42011e6e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86a17c33afae4bef9c8d76a42011e6e2, State: Initialized, Role: FOLLOWER
I20260810 11:35:30.731967 26499 consensus_queue.cc:260] T 00000000000000000000000000000000 P 86a17c33afae4bef9c8d76a42011e6e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86a17c33afae4bef9c8d76a42011e6e2" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 40883 } }
I20260810 11:35:30.732191 26499 raft_consensus.cc:388] T 00000000000000000000000000000000 P 86a17c33afae4bef9c8d76a42011e6e2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 11:35:30.732344 26499 raft_consensus.cc:482] T 00000000000000000000000000000000 P 86a17c33afae4bef9c8d76a42011e6e2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 11:35:30.732503 26499 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 86a17c33afae4bef9c8d76a42011e6e2 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:35:30.735409 26499 raft_consensus.cc:504] T 00000000000000000000000000000000 P 86a17c33afae4bef9c8d76a42011e6e2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86a17c33afae4bef9c8d76a42011e6e2" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 40883 } }
I20260810 11:35:30.735968 26499 leader_election.cc:302] T 00000000000000000000000000000000 P 86a17c33afae4bef9c8d76a42011e6e2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 86a17c33afae4bef9c8d76a42011e6e2; no voters:
I20260810 11:35:30.736616 26499 leader_election.cc:288] T 00000000000000000000000000000000 P 86a17c33afae4bef9c8d76a42011e6e2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 11:35:30.736811 26504 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 86a17c33afae4bef9c8d76a42011e6e2 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:35:30.737846 26504 raft_consensus.cc:686] T 00000000000000000000000000000000 P 86a17c33afae4bef9c8d76a42011e6e2 [term 1 LEADER]: Becoming Leader. State: Replica: 86a17c33afae4bef9c8d76a42011e6e2, State: Running, Role: LEADER
I20260810 11:35:30.738530 26504 consensus_queue.cc:237] T 00000000000000000000000000000000 P 86a17c33afae4bef9c8d76a42011e6e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86a17c33afae4bef9c8d76a42011e6e2" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 40883 } }
I20260810 11:35:30.739259 26499 sys_catalog.cc:564] T 00000000000000000000000000000000 P 86a17c33afae4bef9c8d76a42011e6e2 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 11:35:30.741819 26505 sys_catalog.cc:455] T 00000000000000000000000000000000 P 86a17c33afae4bef9c8d76a42011e6e2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "86a17c33afae4bef9c8d76a42011e6e2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86a17c33afae4bef9c8d76a42011e6e2" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 40883 } } }
I20260810 11:35:30.742084 26506 sys_catalog.cc:455] T 00000000000000000000000000000000 P 86a17c33afae4bef9c8d76a42011e6e2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 86a17c33afae4bef9c8d76a42011e6e2. Latest consensus state: current_term: 1 leader_uuid: "86a17c33afae4bef9c8d76a42011e6e2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86a17c33afae4bef9c8d76a42011e6e2" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 40883 } } }
I20260810 11:35:30.742357 26506 sys_catalog.cc:458] T 00000000000000000000000000000000 P 86a17c33afae4bef9c8d76a42011e6e2 [sys.catalog]: This master's current role is: LEADER
I20260810 11:35:30.742357 26505 sys_catalog.cc:458] T 00000000000000000000000000000000 P 86a17c33afae4bef9c8d76a42011e6e2 [sys.catalog]: This master's current role is: LEADER
I20260810 11:35:30.746644 26512 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 11:35:30.752179 26512 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 11:35:30.763227 26512 catalog_manager.cc:1347] Generated new cluster ID: 2d5f8be5ce5d475493a9b131c8d19899
I20260810 11:35:30.763439 26512 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 11:35:30.772389 26512 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 11:35:30.774166 26512 catalog_manager.cc:1504] Loading token signing keys...
I20260810 11:35:30.785027 26512 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 86a17c33afae4bef9c8d76a42011e6e2: Generated new TSK 0
I20260810 11:35:30.786046 26512 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 11:35:30.801065 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786361401922362-8267-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786361401922362-8267-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786361401922362-8267-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786361401922362-8267-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.18.193:0
--local_ip_for_outbound_sockets=127.8.18.193
--webserver_interface=127.8.18.193
--webserver_port=0
--tserver_master_addrs=127.8.18.254:40883
--builtin_ntp_servers=127.8.18.212:35867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 11:35:31.176283 26523 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:35:31.176611 26523 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:35:31.176705 26523 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:35:31.184304 26523 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:35:31.184439 26523 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:35:31.184554 26523 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.18.193
I20260810 11:35:31.194316 26523 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.18.212:35867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786361401922362-8267-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786361401922362-8267-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.18.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786361401922362-8267-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.18.193
--webserver_port=0
--tserver_master_addrs=127.8.18.254:40883
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.18.193
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786361401922362-8267-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:35:31.195832 26523 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:35:31.197849 26523 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:35:31.212692 26528 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:35:31.212775 26531 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:35:31.212744 26529 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:35:31.215128 26523 server_base.cc:1034] running on GCE node
I20260810 11:35:31.215839 26523 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:35:31.216917 26523 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:35:31.218137 26523 hybrid_clock.cc:648] HybridClock initialized: now 1786361731218084 us; error 37 us; skew 500 ppm
I20260810 11:35:31.218495 26523 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 11:35:31.220984 26523 webserver.cc:460] Webserver started at http://127.8.18.193:34553/ using document root <none> and password file <none>
I20260810 11:35:31.221832 26523 fs_manager.cc:359] Metadata directory not provided
I20260810 11:35:31.221962 26523 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:35:31.222352 26523 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:35:31.225814 26523 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786361401922362-8267-0/minicluster-data/ts-0/data/instance:
uuid: "df4402bb11b84a16a0609b3a2373dc79"
format_stamp: "Formatted at 2026-08-10 11:35:31 on dist-test-slave-srgv"
server_key: "cff7276223d0d67d072829d80a36bcb4"
server_key_iv: "368d7f6e6d0f3a7121ad369a89e3dfb5"
server_key_version: "encryptionkey@0"
I20260810 11:35:31.226766 26523 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786361401922362-8267-0/minicluster-data/ts-0/wal/instance:
uuid: "df4402bb11b84a16a0609b3a2373dc79"
format_stamp: "Formatted at 2026-08-10 11:35:31 on dist-test-slave-srgv"
server_key: "cff7276223d0d67d072829d80a36bcb4"
server_key_iv: "368d7f6e6d0f3a7121ad369a89e3dfb5"
server_key_version: "encryptionkey@0"
I20260810 11:35:31.232817 26523 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260810 11:35:31.236884 26537 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:35:31.238623 26523 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 11:35:31.238808 26523 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786361401922362-8267-0/minicluster-data/ts-0/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786361401922362-8267-0/minicluster-data/ts-0/wal
uuid: "df4402bb11b84a16a0609b3a2373dc79"
format_stamp: "Formatted at 2026-08-10 11:35:31 on dist-test-slave-srgv"
server_key: "cff7276223d0d67d072829d80a36bcb4"
server_key_iv: "368d7f6e6d0f3a7121ad369a89e3dfb5"
server_key_version: "encryptionkey@0"
I20260810 11:35:31.239037 26523 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786361401922362-8267-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786361401922362-8267-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786361401922362-8267-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:35:31.258976 26523 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:35:31.259917 26523 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:35:31.260267 26523 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:35:31.261461 26523 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:35:31.263072 26523 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:35:31.263206 26523 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:35:31.263347 26523 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:35:31.263445 26523 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:35:31.301388 26523 rpc_server.cc:307] RPC server started. Bound to: 127.8.18.193:44019
I20260810 11:35:31.301537 26652 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.18.193:44019 every 8 connection(s)
I20260810 11:35:31.302860 26523 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786361401922362-8267-0/minicluster-data/ts-0/data/info.pb
I20260810 11:35:31.304503 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 26523
I20260810 11:35:31.304699 8267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786361401922362-8267-0/minicluster-data/ts-0/wal/instance
I20260810 11:35:31.305306 8267 external_mini_cluster.cc:1428] Setting key e5dd0d4809fafc572d0203f2201c969e
I20260810 11:35:31.308521 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786361401922362-8267-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786361401922362-8267-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786361401922362-8267-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786361401922362-8267-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.18.194:0
--local_ip_for_outbound_sockets=127.8.18.194
--webserver_interface=127.8.18.194
--webserver_port=0
--tserver_master_addrs=127.8.18.254:40883
--builtin_ntp_servers=127.8.18.212:35867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260810 11:35:31.316910 26655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:31.305856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.193:58107 (local address 127.8.18.254:40883)
0810 11:35:31.306377 (+ 521us) server_negotiation.cc:206] Beginning negotiation
0810 11:35:31.306383 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:35:31.307905 (+ 1522us) server_negotiation.cc:407] Connection header received
0810 11:35:31.309988 (+ 2083us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:35:31.309994 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:35:31.310093 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:35:31.310226 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:35:31.311383 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:31.312127 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:31.313881 (+ 1754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:31.314321 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:31.314467 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:35:31.315995 (+ 1528us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:35:31.316000 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:35:31.316004 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:35:31.316024 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:35:31.316067 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:35:31.316080 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:35:31.316180 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0810 11:35:31.316424 (+ 244us) server_negotiation.cc:299] Negotiation successful
0810 11:35:31.316539 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":407,"thread_start_us":252,"threads_started":1}
I20260810 11:35:31.318068 26656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:31.306589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:40883 (local address 127.8.18.193:58107)
0810 11:35:31.307748 (+ 1159us) negotiation.cc:107] Waiting for socket to connect
0810 11:35:31.307787 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0810 11:35:31.309687 (+ 1900us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:35:31.310476 (+ 789us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:35:31.310495 (+ 19us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:35:31.310591 (+ 96us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:35:31.311193 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:35:31.311212 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:35:31.312311 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:35:31.312315 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:35:31.313726 (+ 1411us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:35:31.313739 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:35:31.314570 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:35:31.314577 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:35:31.314826 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:35:31.315619 (+ 793us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 11:35:31.315622 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:35:31.315668 (+ 46us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 11:35:31.315778 (+ 110us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:35:31.316182 (+ 404us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:35:31.316197 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:35:31.316206 (+ 9us) client_negotiation.cc:769] Sending connection context
0810 11:35:31.316417 (+ 211us) client_negotiation.cc:240] Negotiation successful
0810 11:35:31.316864 (+ 447us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":591,"thread_start_us":372,"threads_started":1}
I20260810 11:35:31.319818 26653 heartbeater.cc:347] Connected to a master server at 127.8.18.254:40883
I20260810 11:35:31.320153 26653 heartbeater.cc:464] Registering TS with master...
I20260810 11:35:31.321130 26653 heartbeater.cc:511] Master 127.8.18.254:40883 requested a full tablet report, sending...
I20260810 11:35:31.323460 26464 ts_manager.cc:194] Registered new tserver with Master: df4402bb11b84a16a0609b3a2373dc79 (127.8.18.193:44019)
I20260810 11:35:31.325022 26464 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.18.193:58107
W20260810 11:35:31.689394 26657 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:35:31.689934 26657 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:35:31.690128 26657 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:35:31.698243 26657 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:35:31.698405 26657 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:35:31.698599 26657 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.18.194
I20260810 11:35:31.708515 26657 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.18.212:35867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786361401922362-8267-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786361401922362-8267-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.18.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786361401922362-8267-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.18.194
--webserver_port=0
--tserver_master_addrs=127.8.18.254:40883
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.18.194
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786361401922362-8267-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:35:31.710063 26657 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:35:31.712086 26657 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:35:31.726198 26662 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:35:31.726292 26663 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:35:31.726217 26665 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:35:31.727177 26657 server_base.cc:1034] running on GCE node
I20260810 11:35:31.727885 26657 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:35:31.728978 26657 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:35:31.730265 26657 hybrid_clock.cc:648] HybridClock initialized: now 1786361731730209 us; error 54 us; skew 500 ppm
I20260810 11:35:31.730665 26657 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 11:35:31.733471 26657 webserver.cc:460] Webserver started at http://127.8.18.194:42801/ using document root <none> and password file <none>
I20260810 11:35:31.734323 26657 fs_manager.cc:359] Metadata directory not provided
I20260810 11:35:31.734480 26657 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:35:31.734851 26657 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:35:31.738336 26657 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786361401922362-8267-0/minicluster-data/ts-1/data/instance:
uuid: "d4af19f6c22946d095eba3410c57bcf7"
format_stamp: "Formatted at 2026-08-10 11:35:31 on dist-test-slave-srgv"
server_key: "3b33acdd34ba06af62ede4bb9159639f"
server_key_iv: "43a549ec7bf7dfb8fc63382c26627ab7"
server_key_version: "encryptionkey@0"
I20260810 11:35:31.739310 26657 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786361401922362-8267-0/minicluster-data/ts-1/wal/instance:
uuid: "d4af19f6c22946d095eba3410c57bcf7"
format_stamp: "Formatted at 2026-08-10 11:35:31 on dist-test-slave-srgv"
server_key: "3b33acdd34ba06af62ede4bb9159639f"
server_key_iv: "43a549ec7bf7dfb8fc63382c26627ab7"
server_key_version: "encryptionkey@0"
I20260810 11:35:31.745595 26657 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260810 11:35:31.749801 26671 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:35:31.751389 26657 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 11:35:31.751609 26657 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786361401922362-8267-0/minicluster-data/ts-1/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786361401922362-8267-0/minicluster-data/ts-1/wal
uuid: "d4af19f6c22946d095eba3410c57bcf7"
format_stamp: "Formatted at 2026-08-10 11:35:31 on dist-test-slave-srgv"
server_key: "3b33acdd34ba06af62ede4bb9159639f"
server_key_iv: "43a549ec7bf7dfb8fc63382c26627ab7"
server_key_version: "encryptionkey@0"
I20260810 11:35:31.751881 26657 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786361401922362-8267-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786361401922362-8267-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786361401922362-8267-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:35:31.776244 26657 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:35:31.777210 26657 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:35:31.777645 26657 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:35:31.779032 26657 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:35:31.780726 26657 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:35:31.780864 26657 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:35:31.781016 26657 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:35:31.781116 26657 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:35:31.817464 26657 rpc_server.cc:307] RPC server started. Bound to: 127.8.18.194:36019
I20260810 11:35:31.817520 26785 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.18.194:36019 every 8 connection(s)
I20260810 11:35:31.819007 26657 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786361401922362-8267-0/minicluster-data/ts-1/data/info.pb
I20260810 11:35:31.823212 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 26657
I20260810 11:35:31.823427 8267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786361401922362-8267-0/minicluster-data/ts-1/wal/instance
I20260810 11:35:31.824069 8267 external_mini_cluster.cc:1428] Setting key 111986f71e902c8548c7ce91bb7349b5
I20260810 11:35:31.827807 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786361401922362-8267-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786361401922362-8267-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786361401922362-8267-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786361401922362-8267-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.18.195:0
--local_ip_for_outbound_sockets=127.8.18.195
--webserver_interface=127.8.18.195
--webserver_port=0
--tserver_master_addrs=127.8.18.254:40883
--builtin_ntp_servers=127.8.18.212:35867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260810 11:35:31.835532 26788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:31.821939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.194:35947 (local address 127.8.18.254:40883)
0810 11:35:31.822527 (+ 588us) server_negotiation.cc:206] Beginning negotiation
0810 11:35:31.822532 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:35:31.823727 (+ 1195us) server_negotiation.cc:407] Connection header received
0810 11:35:31.826302 (+ 2575us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:35:31.826313 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:35:31.826410 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:35:31.826527 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:35:31.828221 (+ 1694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:31.828970 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:31.831794 (+ 2824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:31.832274 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:31.832413 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:35:31.834477 (+ 2064us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:35:31.834483 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:35:31.834487 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:35:31.834510 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:35:31.834580 (+ 70us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:35:31.834595 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:35:31.834688 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0810 11:35:31.835065 (+ 377us) server_negotiation.cc:299] Negotiation successful
0810 11:35:31.835194 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":477,"thread_start_us":348,"threads_started":1}
I20260810 11:35:31.837100 26789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:31.822412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:40883 (local address 127.8.18.194:35947)
0810 11:35:31.823545 (+ 1133us) negotiation.cc:107] Waiting for socket to connect
0810 11:35:31.823594 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0810 11:35:31.825864 (+ 2270us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:35:31.826867 (+ 1003us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:35:31.826881 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:35:31.827023 (+ 142us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:35:31.827997 (+ 974us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:35:31.828033 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:35:31.829166 (+ 1133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:35:31.829174 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:35:31.831552 (+ 2378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:35:31.831577 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:35:31.832572 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:35:31.832582 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:35:31.832954 (+ 372us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:35:31.833946 (+ 992us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 11:35:31.833952 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:35:31.834025 (+ 73us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 11:35:31.834177 (+ 152us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:35:31.834722 (+ 545us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:35:31.834738 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:35:31.834750 (+ 12us) client_negotiation.cc:769] Sending connection context
0810 11:35:31.834969 (+ 219us) client_negotiation.cc:240] Negotiation successful
0810 11:35:31.835509 (+ 540us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":524,"thread_start_us":332,"threads_started":1}
I20260810 11:35:31.839056 26786 heartbeater.cc:347] Connected to a master server at 127.8.18.254:40883
I20260810 11:35:31.839466 26786 heartbeater.cc:464] Registering TS with master...
I20260810 11:35:31.840554 26786 heartbeater.cc:511] Master 127.8.18.254:40883 requested a full tablet report, sending...
I20260810 11:35:31.842593 26464 ts_manager.cc:194] Registered new tserver with Master: d4af19f6c22946d095eba3410c57bcf7 (127.8.18.194:36019)
I20260810 11:35:31.844087 26464 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.18.194:35947
W20260810 11:35:32.205158 26790 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:35:32.205582 26790 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:35:32.205695 26790 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:35:32.213272 26790 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:35:32.213392 26790 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:35:32.213603 26790 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.18.195
I20260810 11:35:32.223361 26790 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.18.212:35867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786361401922362-8267-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786361401922362-8267-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.18.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786361401922362-8267-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.18.195
--webserver_port=0
--tserver_master_addrs=127.8.18.254:40883
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.18.195
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786361401922362-8267-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:35:32.224705 26790 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:35:32.226606 26790 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:35:32.241026 26798 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:35:32.241348 26796 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:35:32.241631 26795 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:35:32.242808 26790 server_base.cc:1034] running on GCE node
I20260810 11:35:32.243709 26790 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:35:32.244863 26790 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:35:32.246109 26790 hybrid_clock.cc:648] HybridClock initialized: now 1786361732246024 us; error 44 us; skew 500 ppm
I20260810 11:35:32.246531 26790 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 11:35:32.249229 26790 webserver.cc:460] Webserver started at http://127.8.18.195:37665/ using document root <none> and password file <none>
I20260810 11:35:32.250151 26790 fs_manager.cc:359] Metadata directory not provided
I20260810 11:35:32.250294 26790 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:35:32.250679 26790 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:35:32.254034 26790 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786361401922362-8267-0/minicluster-data/ts-2/data/instance:
uuid: "108b8dd3313b43bb8db48abccbb8ddda"
format_stamp: "Formatted at 2026-08-10 11:35:32 on dist-test-slave-srgv"
server_key: "bce50d27502e7a57afd2bc0b23ebb9e3"
server_key_iv: "60cc43282ffb4d187096b0bf7a8b7eee"
server_key_version: "encryptionkey@0"
I20260810 11:35:32.255050 26790 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786361401922362-8267-0/minicluster-data/ts-2/wal/instance:
uuid: "108b8dd3313b43bb8db48abccbb8ddda"
format_stamp: "Formatted at 2026-08-10 11:35:32 on dist-test-slave-srgv"
server_key: "bce50d27502e7a57afd2bc0b23ebb9e3"
server_key_iv: "60cc43282ffb4d187096b0bf7a8b7eee"
server_key_version: "encryptionkey@0"
I20260810 11:35:32.260926 26790 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260810 11:35:32.264889 26804 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:35:32.266552 26790 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260810 11:35:32.266760 26790 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786361401922362-8267-0/minicluster-data/ts-2/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786361401922362-8267-0/minicluster-data/ts-2/wal
uuid: "108b8dd3313b43bb8db48abccbb8ddda"
format_stamp: "Formatted at 2026-08-10 11:35:32 on dist-test-slave-srgv"
server_key: "bce50d27502e7a57afd2bc0b23ebb9e3"
server_key_iv: "60cc43282ffb4d187096b0bf7a8b7eee"
server_key_version: "encryptionkey@0"
I20260810 11:35:32.267017 26790 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786361401922362-8267-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786361401922362-8267-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786361401922362-8267-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:35:32.294075 26790 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:35:32.295066 26790 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:35:32.295447 26790 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:35:32.296581 26790 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:35:32.298269 26790 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:35:32.298362 26790 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:35:32.298503 26790 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:35:32.298595 26790 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:35:32.329497 26653 heartbeater.cc:503] Master 127.8.18.254:40883 was elected leader, sending a full tablet report...
I20260810 11:35:32.335564 26790 rpc_server.cc:307] RPC server started. Bound to: 127.8.18.195:32913
I20260810 11:35:32.335682 26918 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.18.195:32913 every 8 connection(s)
I20260810 11:35:32.336992 26790 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786361401922362-8267-0/minicluster-data/ts-2/data/info.pb
I20260810 11:35:32.338071 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 26790
I20260810 11:35:32.338259 8267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786361401922362-8267-0/minicluster-data/ts-2/wal/instance
I20260810 11:35:32.338869 8267 external_mini_cluster.cc:1428] Setting key 96cf270d7a04507d85f8962109c193c9
I20260810 11:35:32.351397 26921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:32.340245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.195:43597 (local address 127.8.18.254:40883)
0810 11:35:32.340759 (+ 514us) server_negotiation.cc:206] Beginning negotiation
0810 11:35:32.340763 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 11:35:32.342306 (+ 1543us) server_negotiation.cc:407] Connection header received
0810 11:35:32.344428 (+ 2122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:35:32.344434 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:35:32.344527 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:35:32.344668 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:35:32.345866 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:32.346652 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:32.348361 (+ 1709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:32.348788 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:32.348930 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:35:32.350459 (+ 1529us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:35:32.350465 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:35:32.350468 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:35:32.350488 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:35:32.350534 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:35:32.350546 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:35:32.350653 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0810 11:35:32.350894 (+ 241us) server_negotiation.cc:299] Negotiation successful
0810 11:35:32.351006 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":391,"thread_start_us":261,"threads_started":1}
I20260810 11:35:32.352412 26922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:32.340961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:40883 (local address 127.8.18.195:43597)
0810 11:35:32.342134 (+ 1173us) negotiation.cc:107] Waiting for socket to connect
0810 11:35:32.342193 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0810 11:35:32.344142 (+ 1949us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:35:32.344915 (+ 773us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:35:32.344925 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:35:32.345026 (+ 101us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:35:32.345692 (+ 666us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:35:32.345711 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:35:32.346829 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:35:32.346833 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:35:32.348216 (+ 1383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:35:32.348229 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:35:32.349036 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:35:32.349044 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:35:32.349287 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:35:32.350103 (+ 816us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 11:35:32.350106 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:35:32.350147 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 11:35:32.350235 (+ 88us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:35:32.350653 (+ 418us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:35:32.350667 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:35:32.350677 (+ 10us) client_negotiation.cc:769] Sending connection context
0810 11:35:32.350866 (+ 189us) client_negotiation.cc:240] Negotiation successful
0810 11:35:32.351313 (+ 447us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":572,"thread_start_us":310,"threads_started":1}
I20260810 11:35:32.353973 26919 heartbeater.cc:347] Connected to a master server at 127.8.18.254:40883
I20260810 11:35:32.354344 26919 heartbeater.cc:464] Registering TS with master...
I20260810 11:35:32.355285 26919 heartbeater.cc:511] Master 127.8.18.254:40883 requested a full tablet report, sending...
I20260810 11:35:32.356904 26463 ts_manager.cc:194] Registered new tserver with Master: 108b8dd3313b43bb8db48abccbb8ddda (127.8.18.195:32913)
I20260810 11:35:32.357921 26463 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.18.195:43597
I20260810 11:35:32.367691 8267 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 11:35:32.378871 26921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:32.372229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59086 (local address 127.8.18.254:40883)
0810 11:35:32.372473 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0810 11:35:32.372480 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:35:32.372780 (+ 300us) server_negotiation.cc:407] Connection header received
0810 11:35:32.372992 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:35:32.372998 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:35:32.373084 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:35:32.373197 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:35:32.373855 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:32.374595 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:32.376092 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:32.376537 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:32.376695 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:35:32.378030 (+ 1335us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:35:32.378038 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:35:32.378042 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:35:32.378068 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:35:32.378107 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:35:32.378121 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:35:32.378237 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0810 11:35:32.378406 (+ 169us) server_negotiation.cc:299] Negotiation successful
0810 11:35:32.378550 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260810 11:35:32.383953 26463 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59086:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 11:35:32.391459 26463 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 11:35:32.439806 26935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:32.406769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58818 (local address 127.8.18.194:36019)
0810 11:35:32.431544 (+ 24775us) server_negotiation.cc:206] Beginning negotiation
0810 11:35:32.431552 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:35:32.431577 (+ 25us) server_negotiation.cc:407] Connection header received
0810 11:35:32.431658 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:35:32.431673 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:35:32.431846 (+ 173us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:35:32.432860 (+ 1014us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:35:32.433083 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:32.434663 (+ 1580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:32.437037 (+ 2374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:32.438341 (+ 1304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:32.438537 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:35:32.438863 (+ 326us) server_negotiation.cc:1035] Waiting for connection context
0810 11:35:32.439182 (+ 319us) server_negotiation.cc:299] Negotiation successful
0810 11:35:32.439369 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24593,"thread_start_us":24455,"threads_started":1}
I20260810 11:35:32.439994 26933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:32.406245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.194:36019 (local address 127.0.0.1:58818)
0810 11:35:32.421873 (+ 15628us) negotiation.cc:107] Waiting for socket to connect
0810 11:35:32.421958 (+ 85us) client_negotiation.cc:174] Beginning negotiation
0810 11:35:32.422281 (+ 323us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:35:32.432125 (+ 9844us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:35:32.432131 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:35:32.432167 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:35:32.432646 (+ 479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:35:32.432659 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:35:32.434829 (+ 2170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:35:32.434836 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:35:32.436782 (+ 1946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:35:32.436800 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:35:32.438622 (+ 1822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:35:32.438629 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:35:32.438877 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:35:32.438946 (+ 69us) client_negotiation.cc:769] Sending connection context
0810 11:35:32.439122 (+ 176us) client_negotiation.cc:240] Negotiation successful
0810 11:35:32.439365 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15220,"mutex_wait_us":31,"thread_start_us":16319,"threads_started":1}
I20260810 11:35:32.440697 26938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:32.407641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54404 (local address 127.8.18.195:32913)
0810 11:35:32.426688 (+ 19047us) server_negotiation.cc:206] Beginning negotiation
0810 11:35:32.426701 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 11:35:32.426755 (+ 54us) server_negotiation.cc:407] Connection header received
0810 11:35:32.426908 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:35:32.426921 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:35:32.427294 (+ 373us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:35:32.427605 (+ 311us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:35:32.431583 (+ 3978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:32.433842 (+ 2259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:32.436807 (+ 2965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:32.438530 (+ 1723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:32.438688 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:35:32.438982 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0810 11:35:32.440097 (+ 1115us) server_negotiation.cc:299] Negotiation successful
0810 11:35:32.440278 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18839,"thread_start_us":18667,"threads_started":1}
I20260810 11:35:32.441208 26934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:32.406736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45680 (local address 127.8.18.193:44019)
0810 11:35:32.430260 (+ 23524us) server_negotiation.cc:206] Beginning negotiation
0810 11:35:32.430267 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:35:32.430295 (+ 28us) server_negotiation.cc:407] Connection header received
0810 11:35:32.430378 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:35:32.430384 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:35:32.430617 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:35:32.430847 (+ 230us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:35:32.431551 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:32.433217 (+ 1666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:32.436482 (+ 3265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:32.438511 (+ 2029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:32.438681 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:35:32.439001 (+ 320us) server_negotiation.cc:1035] Waiting for connection context
0810 11:35:32.439849 (+ 848us) server_negotiation.cc:299] Negotiation successful
0810 11:35:32.440573 (+ 724us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23351,"thread_start_us":23202,"threads_started":1}
I20260810 11:35:32.441267 26937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:32.406739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.195:32913 (local address 127.0.0.1:54404)
0810 11:35:32.425760 (+ 19021us) negotiation.cc:107] Waiting for socket to connect
0810 11:35:32.425792 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0810 11:35:32.425993 (+ 201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:35:32.428135 (+ 2142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:35:32.428148 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:35:32.428202 (+ 54us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:35:32.428796 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:35:32.428816 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:35:32.434063 (+ 5247us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:35:32.434068 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:35:32.436491 (+ 2423us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:35:32.436505 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:35:32.439149 (+ 2644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:35:32.439159 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:35:32.439532 (+ 373us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:35:32.439625 (+ 93us) client_negotiation.cc:769] Sending connection context
0810 11:35:32.439723 (+ 98us) client_negotiation.cc:240] Negotiation successful
0810 11:35:32.439890 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18810,"mutex_wait_us":250,"thread_start_us":22422,"threads_started":1}
I20260810 11:35:32.441967 26936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:32.406958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.193:44019 (local address 127.0.0.1:45680)
0810 11:35:32.429562 (+ 22604us) negotiation.cc:107] Waiting for socket to connect
0810 11:35:32.429589 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0810 11:35:32.429708 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:35:32.430839 (+ 1131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:35:32.430844 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:35:32.430871 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:35:32.431299 (+ 428us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:35:32.431314 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:35:32.433398 (+ 2084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:35:32.433431 (+ 33us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:35:32.436121 (+ 2690us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:35:32.436158 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:35:32.439196 (+ 3038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:35:32.439203 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:35:32.439593 (+ 390us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:35:32.439642 (+ 49us) client_negotiation.cc:769] Sending connection context
0810 11:35:32.439721 (+ 79us) client_negotiation.cc:240] Negotiation successful
0810 11:35:32.441608 (+ 1887us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22417,"mutex_wait_us":1775,"thread_start_us":18377,"threads_started":1}
I20260810 11:35:32.445896 26721 tablet_service.cc:1467] Processing CreateTablet for tablet ab5067140d374c7cb605dd98198a9ccc (DEFAULT_TABLE table=test-table [id=21bd3243147640c7a5035a795f4a817b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:35:32.445888 26587 tablet_service.cc:1467] Processing CreateTablet for tablet ab5067140d374c7cb605dd98198a9ccc (DEFAULT_TABLE table=test-table [id=21bd3243147640c7a5035a795f4a817b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:35:32.447583 26721 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab5067140d374c7cb605dd98198a9ccc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:35:32.447597 26587 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab5067140d374c7cb605dd98198a9ccc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:35:32.456355 26852 tablet_service.cc:1467] Processing CreateTablet for tablet ab5067140d374c7cb605dd98198a9ccc (DEFAULT_TABLE table=test-table [id=21bd3243147640c7a5035a795f4a817b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:35:32.457927 26852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab5067140d374c7cb605dd98198a9ccc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:35:32.463893 26939 tablet_bootstrap.cc:492] T ab5067140d374c7cb605dd98198a9ccc P df4402bb11b84a16a0609b3a2373dc79: Bootstrap starting.
I20260810 11:35:32.464809 26940 tablet_bootstrap.cc:492] T ab5067140d374c7cb605dd98198a9ccc P d4af19f6c22946d095eba3410c57bcf7: Bootstrap starting.
I20260810 11:35:32.469523 26939 tablet_bootstrap.cc:654] T ab5067140d374c7cb605dd98198a9ccc P df4402bb11b84a16a0609b3a2373dc79: Neither blocks nor log segments found. Creating new log.
I20260810 11:35:32.471135 26939 log.cc:824] T ab5067140d374c7cb605dd98198a9ccc P df4402bb11b84a16a0609b3a2373dc79: Log is configured to *not* fsync() on all Append() calls
I20260810 11:35:32.473080 26940 tablet_bootstrap.cc:654] T ab5067140d374c7cb605dd98198a9ccc P d4af19f6c22946d095eba3410c57bcf7: Neither blocks nor log segments found. Creating new log.
I20260810 11:35:32.473894 26942 tablet_bootstrap.cc:492] T ab5067140d374c7cb605dd98198a9ccc P 108b8dd3313b43bb8db48abccbb8ddda: Bootstrap starting.
I20260810 11:35:32.474516 26940 log.cc:824] T ab5067140d374c7cb605dd98198a9ccc P d4af19f6c22946d095eba3410c57bcf7: Log is configured to *not* fsync() on all Append() calls
I20260810 11:35:32.474885 26939 tablet_bootstrap.cc:492] T ab5067140d374c7cb605dd98198a9ccc P df4402bb11b84a16a0609b3a2373dc79: No bootstrap required, opened a new log
I20260810 11:35:32.475195 26939 ts_tablet_manager.cc:1397] T ab5067140d374c7cb605dd98198a9ccc P df4402bb11b84a16a0609b3a2373dc79: Time spent bootstrapping tablet: real 0.012s user 0.004s sys 0.005s
I20260810 11:35:32.477789 26942 tablet_bootstrap.cc:654] T ab5067140d374c7cb605dd98198a9ccc P 108b8dd3313b43bb8db48abccbb8ddda: Neither blocks nor log segments found. Creating new log.
I20260810 11:35:32.479135 26942 log.cc:824] T ab5067140d374c7cb605dd98198a9ccc P 108b8dd3313b43bb8db48abccbb8ddda: Log is configured to *not* fsync() on all Append() calls
I20260810 11:35:32.480234 26940 tablet_bootstrap.cc:492] T ab5067140d374c7cb605dd98198a9ccc P d4af19f6c22946d095eba3410c57bcf7: No bootstrap required, opened a new log
I20260810 11:35:32.480581 26940 ts_tablet_manager.cc:1397] T ab5067140d374c7cb605dd98198a9ccc P d4af19f6c22946d095eba3410c57bcf7: Time spent bootstrapping tablet: real 0.016s user 0.005s sys 0.005s
I20260810 11:35:32.482798 26942 tablet_bootstrap.cc:492] T ab5067140d374c7cb605dd98198a9ccc P 108b8dd3313b43bb8db48abccbb8ddda: No bootstrap required, opened a new log
I20260810 11:35:32.483088 26942 ts_tablet_manager.cc:1397] T ab5067140d374c7cb605dd98198a9ccc P 108b8dd3313b43bb8db48abccbb8ddda: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.006s
I20260810 11:35:32.484988 26939 raft_consensus.cc:348] T ab5067140d374c7cb605dd98198a9ccc P df4402bb11b84a16a0609b3a2373dc79 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df4402bb11b84a16a0609b3a2373dc79" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 44019 } } peers { permanent_uuid: "d4af19f6c22946d095eba3410c57bcf7" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 36019 } } peers { permanent_uuid: "108b8dd3313b43bb8db48abccbb8ddda" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 32913 } }
I20260810 11:35:32.485450 26939 raft_consensus.cc:374] T ab5067140d374c7cb605dd98198a9ccc P df4402bb11b84a16a0609b3a2373dc79 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:35:32.485594 26939 raft_consensus.cc:729] T ab5067140d374c7cb605dd98198a9ccc P df4402bb11b84a16a0609b3a2373dc79 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df4402bb11b84a16a0609b3a2373dc79, State: Initialized, Role: FOLLOWER
I20260810 11:35:32.486264 26939 consensus_queue.cc:260] T ab5067140d374c7cb605dd98198a9ccc P df4402bb11b84a16a0609b3a2373dc79 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df4402bb11b84a16a0609b3a2373dc79" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 44019 } } peers { permanent_uuid: "d4af19f6c22946d095eba3410c57bcf7" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 36019 } } peers { permanent_uuid: "108b8dd3313b43bb8db48abccbb8ddda" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 32913 } }
I20260810 11:35:32.488188 26939 ts_tablet_manager.cc:1428] T ab5067140d374c7cb605dd98198a9ccc P df4402bb11b84a16a0609b3a2373dc79: Time spent starting tablet: real 0.013s user 0.004s sys 0.008s
I20260810 11:35:32.491858 26940 raft_consensus.cc:348] T ab5067140d374c7cb605dd98198a9ccc P d4af19f6c22946d095eba3410c57bcf7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df4402bb11b84a16a0609b3a2373dc79" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 44019 } } peers { permanent_uuid: "d4af19f6c22946d095eba3410c57bcf7" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 36019 } } peers { permanent_uuid: "108b8dd3313b43bb8db48abccbb8ddda" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 32913 } }
I20260810 11:35:32.492373 26940 raft_consensus.cc:374] T ab5067140d374c7cb605dd98198a9ccc P d4af19f6c22946d095eba3410c57bcf7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:35:32.492463 26940 raft_consensus.cc:729] T ab5067140d374c7cb605dd98198a9ccc P d4af19f6c22946d095eba3410c57bcf7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4af19f6c22946d095eba3410c57bcf7, State: Initialized, Role: FOLLOWER
I20260810 11:35:32.493147 26940 consensus_queue.cc:260] T ab5067140d374c7cb605dd98198a9ccc P d4af19f6c22946d095eba3410c57bcf7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df4402bb11b84a16a0609b3a2373dc79" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 44019 } } peers { permanent_uuid: "d4af19f6c22946d095eba3410c57bcf7" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 36019 } } peers { permanent_uuid: "108b8dd3313b43bb8db48abccbb8ddda" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 32913 } }
I20260810 11:35:32.494102 26942 raft_consensus.cc:348] T ab5067140d374c7cb605dd98198a9ccc P 108b8dd3313b43bb8db48abccbb8ddda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df4402bb11b84a16a0609b3a2373dc79" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 44019 } } peers { permanent_uuid: "d4af19f6c22946d095eba3410c57bcf7" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 36019 } } peers { permanent_uuid: "108b8dd3313b43bb8db48abccbb8ddda" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 32913 } }
I20260810 11:35:32.494504 26786 heartbeater.cc:503] Master 127.8.18.254:40883 was elected leader, sending a full tablet report...
I20260810 11:35:32.494561 26942 raft_consensus.cc:374] T ab5067140d374c7cb605dd98198a9ccc P 108b8dd3313b43bb8db48abccbb8ddda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:35:32.494660 26942 raft_consensus.cc:729] T ab5067140d374c7cb605dd98198a9ccc P 108b8dd3313b43bb8db48abccbb8ddda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 108b8dd3313b43bb8db48abccbb8ddda, State: Initialized, Role: FOLLOWER
I20260810 11:35:32.495275 26940 ts_tablet_manager.cc:1428] T ab5067140d374c7cb605dd98198a9ccc P d4af19f6c22946d095eba3410c57bcf7: Time spent starting tablet: real 0.015s user 0.011s sys 0.005s
I20260810 11:35:32.495440 26942 consensus_queue.cc:260] T ab5067140d374c7cb605dd98198a9ccc P 108b8dd3313b43bb8db48abccbb8ddda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df4402bb11b84a16a0609b3a2373dc79" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 44019 } } peers { permanent_uuid: "d4af19f6c22946d095eba3410c57bcf7" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 36019 } } peers { permanent_uuid: "108b8dd3313b43bb8db48abccbb8ddda" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 32913 } }
I20260810 11:35:32.496606 26919 heartbeater.cc:503] Master 127.8.18.254:40883 was elected leader, sending a full tablet report...
I20260810 11:35:32.497392 26942 ts_tablet_manager.cc:1428] T ab5067140d374c7cb605dd98198a9ccc P 108b8dd3313b43bb8db48abccbb8ddda: Time spent starting tablet: real 0.014s user 0.012s sys 0.003s
I20260810 11:35:32.529799 26947 raft_consensus.cc:482] T ab5067140d374c7cb605dd98198a9ccc P 108b8dd3313b43bb8db48abccbb8ddda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 11:35:32.530299 26947 raft_consensus.cc:504] T ab5067140d374c7cb605dd98198a9ccc P 108b8dd3313b43bb8db48abccbb8ddda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df4402bb11b84a16a0609b3a2373dc79" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 44019 } } peers { permanent_uuid: "d4af19f6c22946d095eba3410c57bcf7" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 36019 } } peers { permanent_uuid: "108b8dd3313b43bb8db48abccbb8ddda" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 32913 } }
I20260810 11:35:32.532368 26947 leader_election.cc:288] T ab5067140d374c7cb605dd98198a9ccc P 108b8dd3313b43bb8db48abccbb8ddda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers df4402bb11b84a16a0609b3a2373dc79 (127.8.18.193:44019), d4af19f6c22946d095eba3410c57bcf7 (127.8.18.194:36019)
I20260810 11:35:32.539171 26934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:32.532996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.195:47107 (local address 127.8.18.193:44019)
0810 11:35:32.533261 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0810 11:35:32.533267 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:35:32.533290 (+ 23us) server_negotiation.cc:407] Connection header received
0810 11:35:32.533374 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:35:32.533381 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:35:32.533506 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:35:32.533623 (+ 117us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:35:32.534258 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:32.535225 (+ 967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:32.537101 (+ 1876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:32.538145 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:32.538269 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:35:32.538321 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0810 11:35:32.538759 (+ 438us) server_negotiation.cc:299] Negotiation successful
0810 11:35:32.538909 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105}
I20260810 11:35:32.539213 26922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:32.532668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.193:44019 (local address 127.8.18.195:47107)
0810 11:35:32.532895 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0810 11:35:32.532919 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0810 11:35:32.533050 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:35:32.533697 (+ 647us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:35:32.533704 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:35:32.533730 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:35:32.534041 (+ 311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:35:32.534054 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:35:32.535375 (+ 1321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:35:32.535381 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:35:32.536921 (+ 1540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:35:32.536937 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:35:32.538309 (+ 1372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:35:32.538317 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:35:32.538524 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:35:32.538564 (+ 40us) client_negotiation.cc:769] Sending connection context
0810 11:35:32.538671 (+ 107us) client_negotiation.cc:240] Negotiation successful
0810 11:35:32.538828 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":79,"spinlock_wait_cycles":39552}
I20260810 11:35:32.539899 26935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:32.532866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.195:37713 (local address 127.8.18.194:36019)
0810 11:35:32.533184 (+ 318us) server_negotiation.cc:206] Beginning negotiation
0810 11:35:32.533193 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 11:35:32.533844 (+ 651us) server_negotiation.cc:407] Connection header received
0810 11:35:32.534007 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:35:32.534013 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:35:32.534111 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:35:32.534208 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:35:32.534825 (+ 617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:32.535698 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:32.537687 (+ 1989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:32.538694 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:32.538825 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:35:32.538888 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0810 11:35:32.539370 (+ 482us) server_negotiation.cc:299] Negotiation successful
0810 11:35:32.539543 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260810 11:35:32.540452 26948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:32.532668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.194:36019 (local address 127.8.18.195:37713)
0810 11:35:32.533717 (+ 1049us) negotiation.cc:107] Waiting for socket to connect
0810 11:35:32.533735 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 11:35:32.533874 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:35:32.534252 (+ 378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:35:32.534256 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:35:32.534281 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:35:32.534654 (+ 373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:35:32.534667 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:35:32.535861 (+ 1194us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:35:32.535865 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:35:32.537487 (+ 1622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:35:32.537504 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:35:32.538907 (+ 1403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:35:32.538913 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:35:32.539154 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:35:32.539193 (+ 39us) client_negotiation.cc:769] Sending connection context
0810 11:35:32.539900 (+ 707us) client_negotiation.cc:240] Negotiation successful
0810 11:35:32.540075 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":909,"mutex_wait_us":159,"spinlock_wait_cycles":768,"thread_start_us":459,"threads_started":1}
I20260810 11:35:32.540423 26607 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab5067140d374c7cb605dd98198a9ccc" candidate_uuid: "108b8dd3313b43bb8db48abccbb8ddda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df4402bb11b84a16a0609b3a2373dc79" is_pre_election: true
I20260810 11:35:32.541095 26607 raft_consensus.cc:2454] T ab5067140d374c7cb605dd98198a9ccc P df4402bb11b84a16a0609b3a2373dc79 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 108b8dd3313b43bb8db48abccbb8ddda in term 0.
I20260810 11:35:32.541637 26741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab5067140d374c7cb605dd98198a9ccc" candidate_uuid: "108b8dd3313b43bb8db48abccbb8ddda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4af19f6c22946d095eba3410c57bcf7" is_pre_election: true
I20260810 11:35:32.541887 26805 leader_election.cc:302] T ab5067140d374c7cb605dd98198a9ccc P 108b8dd3313b43bb8db48abccbb8ddda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 108b8dd3313b43bb8db48abccbb8ddda, df4402bb11b84a16a0609b3a2373dc79; no voters:
I20260810 11:35:32.542191 26741 raft_consensus.cc:2454] T ab5067140d374c7cb605dd98198a9ccc P d4af19f6c22946d095eba3410c57bcf7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 108b8dd3313b43bb8db48abccbb8ddda in term 0.
I20260810 11:35:32.542390 26947 raft_consensus.cc:2781] T ab5067140d374c7cb605dd98198a9ccc P 108b8dd3313b43bb8db48abccbb8ddda [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 11:35:32.542553 26947 raft_consensus.cc:482] T ab5067140d374c7cb605dd98198a9ccc P 108b8dd3313b43bb8db48abccbb8ddda [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 11:35:32.542694 26947 raft_consensus.cc:3037] T ab5067140d374c7cb605dd98198a9ccc P 108b8dd3313b43bb8db48abccbb8ddda [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:35:32.545257 26947 raft_consensus.cc:504] T ab5067140d374c7cb605dd98198a9ccc P 108b8dd3313b43bb8db48abccbb8ddda [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df4402bb11b84a16a0609b3a2373dc79" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 44019 } } peers { permanent_uuid: "d4af19f6c22946d095eba3410c57bcf7" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 36019 } } peers { permanent_uuid: "108b8dd3313b43bb8db48abccbb8ddda" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 32913 } }
I20260810 11:35:32.545933 26947 leader_election.cc:288] T ab5067140d374c7cb605dd98198a9ccc P 108b8dd3313b43bb8db48abccbb8ddda [CANDIDATE]: Term 1 election: Requested vote from peers df4402bb11b84a16a0609b3a2373dc79 (127.8.18.193:44019), d4af19f6c22946d095eba3410c57bcf7 (127.8.18.194:36019)
I20260810 11:35:32.546548 26607 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab5067140d374c7cb605dd98198a9ccc" candidate_uuid: "108b8dd3313b43bb8db48abccbb8ddda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df4402bb11b84a16a0609b3a2373dc79"
I20260810 11:35:32.546548 26741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab5067140d374c7cb605dd98198a9ccc" candidate_uuid: "108b8dd3313b43bb8db48abccbb8ddda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4af19f6c22946d095eba3410c57bcf7"
I20260810 11:35:32.546813 26741 raft_consensus.cc:3037] T ab5067140d374c7cb605dd98198a9ccc P d4af19f6c22946d095eba3410c57bcf7 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:35:32.546811 26607 raft_consensus.cc:3037] T ab5067140d374c7cb605dd98198a9ccc P df4402bb11b84a16a0609b3a2373dc79 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:35:32.550328 26607 raft_consensus.cc:2454] T ab5067140d374c7cb605dd98198a9ccc P df4402bb11b84a16a0609b3a2373dc79 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 108b8dd3313b43bb8db48abccbb8ddda in term 1.
I20260810 11:35:32.550335 26741 raft_consensus.cc:2454] T ab5067140d374c7cb605dd98198a9ccc P d4af19f6c22946d095eba3410c57bcf7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 108b8dd3313b43bb8db48abccbb8ddda in term 1.
I20260810 11:35:32.550892 26807 leader_election.cc:302] T ab5067140d374c7cb605dd98198a9ccc P 108b8dd3313b43bb8db48abccbb8ddda [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 108b8dd3313b43bb8db48abccbb8ddda, d4af19f6c22946d095eba3410c57bcf7; no voters:
I20260810 11:35:32.551331 26947 raft_consensus.cc:2781] T ab5067140d374c7cb605dd98198a9ccc P 108b8dd3313b43bb8db48abccbb8ddda [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:35:32.551846 26947 raft_consensus.cc:686] T ab5067140d374c7cb605dd98198a9ccc P 108b8dd3313b43bb8db48abccbb8ddda [term 1 LEADER]: Becoming Leader. State: Replica: 108b8dd3313b43bb8db48abccbb8ddda, State: Running, Role: LEADER
I20260810 11:35:32.552358 26947 consensus_queue.cc:237] T ab5067140d374c7cb605dd98198a9ccc P 108b8dd3313b43bb8db48abccbb8ddda [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df4402bb11b84a16a0609b3a2373dc79" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 44019 } } peers { permanent_uuid: "d4af19f6c22946d095eba3410c57bcf7" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 36019 } } peers { permanent_uuid: "108b8dd3313b43bb8db48abccbb8ddda" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 32913 } }
W20260810 11:35:32.555598 26654 tablet.cc:2369] T ab5067140d374c7cb605dd98198a9ccc P df4402bb11b84a16a0609b3a2373dc79: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 11:35:32.557770 26464 catalog_manager.cc:5495] T ab5067140d374c7cb605dd98198a9ccc P 108b8dd3313b43bb8db48abccbb8ddda reported cstate change: term changed from 0 to 1, leader changed from <none> to 108b8dd3313b43bb8db48abccbb8ddda (127.8.18.195). New cstate: current_term: 1 leader_uuid: "108b8dd3313b43bb8db48abccbb8ddda" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df4402bb11b84a16a0609b3a2373dc79" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 44019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d4af19f6c22946d095eba3410c57bcf7" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 36019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "108b8dd3313b43bb8db48abccbb8ddda" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 32913 } health_report { overall_health: HEALTHY } } }
W20260810 11:35:32.571417 26787 tablet.cc:2369] T ab5067140d374c7cb605dd98198a9ccc P d4af19f6c22946d095eba3410c57bcf7: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 11:35:32.588980 26920 tablet.cc:2369] T ab5067140d374c7cb605dd98198a9ccc P 108b8dd3313b43bb8db48abccbb8ddda: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 11:35:32.599284 26938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:32.593792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54420 (local address 127.8.18.195:32913)
0810 11:35:32.593982 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0810 11:35:32.593986 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 11:35:32.594005 (+ 19us) server_negotiation.cc:407] Connection header received
0810 11:35:32.594127 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:35:32.594131 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:35:32.594221 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:35:32.594351 (+ 130us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 11:35:32.594699 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:32.595442 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:32.596903 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:32.597350 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:32.597558 (+ 208us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:35:32.598046 (+ 488us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 11:35:32.598780 (+ 734us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 11:35:32.598957 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0810 11:35:32.599005 (+ 48us) server_negotiation.cc:299] Negotiation successful
0810 11:35:32.599086 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260810 11:35:32.608000 26741 raft_consensus.cc:1261] T ab5067140d374c7cb605dd98198a9ccc P d4af19f6c22946d095eba3410c57bcf7 [term 1 FOLLOWER]: Refusing update from remote peer 108b8dd3313b43bb8db48abccbb8ddda: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 11:35:32.608000 26607 raft_consensus.cc:1261] T ab5067140d374c7cb605dd98198a9ccc P df4402bb11b84a16a0609b3a2373dc79 [term 1 FOLLOWER]: Refusing update from remote peer 108b8dd3313b43bb8db48abccbb8ddda: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 11:35:32.609110 26949 consensus_queue.cc:1035] T ab5067140d374c7cb605dd98198a9ccc P 108b8dd3313b43bb8db48abccbb8ddda [LEADER]: Connected to new peer: Peer: permanent_uuid: "df4402bb11b84a16a0609b3a2373dc79" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 44019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 11:35:32.609601 26947 consensus_queue.cc:1035] T ab5067140d374c7cb605dd98198a9ccc P 108b8dd3313b43bb8db48abccbb8ddda [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4af19f6c22946d095eba3410c57bcf7" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 36019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 11:35:32.628490 26954 mvcc.cc:205] Tried to move back new op lower bound from 7316937656751271936 to 7316937656538931200. Current Snapshot: MvccSnapshot[applied={T|T < 7316937656751271936}]
I20260810 11:35:32.645749 26463 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:59086:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 11:35:32.646121 26463 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:59086:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 11:35:32.650568 26463 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 86a17c33afae4bef9c8d76a42011e6e2: Sending DeleteTablet for 3 replicas of tablet ab5067140d374c7cb605dd98198a9ccc
I20260810 11:35:32.651942 26587 tablet_service.cc:1514] Processing DeleteTablet for tablet ab5067140d374c7cb605dd98198a9ccc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:35:32 UTC) from {username='slave'} at 127.0.0.1:45680
I20260810 11:35:32.651971 26721 tablet_service.cc:1514] Processing DeleteTablet for tablet ab5067140d374c7cb605dd98198a9ccc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:35:32 UTC) from {username='slave'} at 127.0.0.1:58818
I20260810 11:35:32.652202 26852 tablet_service.cc:1514] Processing DeleteTablet for tablet ab5067140d374c7cb605dd98198a9ccc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:35:32 UTC) from {username='slave'} at 127.0.0.1:54404
I20260810 11:35:32.654165 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 26523
I20260810 11:35:32.666100 26965 tablet_replica.cc:330] T ab5067140d374c7cb605dd98198a9ccc P d4af19f6c22946d095eba3410c57bcf7: stopping tablet replica
I20260810 11:35:32.667098 26965 raft_consensus.cc:2229] T ab5067140d374c7cb605dd98198a9ccc P d4af19f6c22946d095eba3410c57bcf7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 11:35:32.667441 26964 tablet_replica.cc:330] T ab5067140d374c7cb605dd98198a9ccc P df4402bb11b84a16a0609b3a2373dc79: stopping tablet replica
I20260810 11:35:32.668076 26964 raft_consensus.cc:2229] T ab5067140d374c7cb605dd98198a9ccc P df4402bb11b84a16a0609b3a2373dc79 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 11:35:32.668918 26964 raft_consensus.cc:2258] T ab5067140d374c7cb605dd98198a9ccc P df4402bb11b84a16a0609b3a2373dc79 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:35:32.669620 26965 raft_consensus.cc:2258] T ab5067140d374c7cb605dd98198a9ccc P d4af19f6c22946d095eba3410c57bcf7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:35:32.673705 26964 ts_tablet_manager.cc:1905] T ab5067140d374c7cb605dd98198a9ccc P df4402bb11b84a16a0609b3a2373dc79: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:35:32.675767 26965 ts_tablet_manager.cc:1905] T ab5067140d374c7cb605dd98198a9ccc P d4af19f6c22946d095eba3410c57bcf7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:35:32.676646 26966 tablet_replica.cc:330] T ab5067140d374c7cb605dd98198a9ccc P 108b8dd3313b43bb8db48abccbb8ddda: stopping tablet replica
I20260810 11:35:32.677243 26966 raft_consensus.cc:2229] T ab5067140d374c7cb605dd98198a9ccc P 108b8dd3313b43bb8db48abccbb8ddda [term 1 LEADER]: Raft consensus shutting down.
I20260810 11:35:32.678159 26966 raft_consensus.cc:2258] T ab5067140d374c7cb605dd98198a9ccc P 108b8dd3313b43bb8db48abccbb8ddda [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:35:32.680398 26934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:32.655756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45690 (local address 127.8.18.193:44019)
0810 11:35:32.655957 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0810 11:35:32.655963 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:35:32.669987 (+ 14024us) server_negotiation.cc:407] Connection header received
0810 11:35:32.670543 (+ 556us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:35:32.670550 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:35:32.670691 (+ 141us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:35:32.670777 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:35:32.672163 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:32.673330 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:32.675798 (+ 2468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:32.676480 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:32.676626 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:35:32.678484 (+ 1858us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:35:32.678497 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:35:32.678515 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:35:32.679351 (+ 836us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:35:32.679419 (+ 68us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:35:32.679447 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:35:32.679546 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0810 11:35:32.679827 (+ 281us) server_negotiation.cc:299] Negotiation successful
0810 11:35:32.679974 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260810 11:35:32.681717 26648 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:35:32.684633 26966 ts_tablet_manager.cc:1905] T ab5067140d374c7cb605dd98198a9ccc P 108b8dd3313b43bb8db48abccbb8ddda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:35:32.693717 26966 ts_tablet_manager.cc:1918] T ab5067140d374c7cb605dd98198a9ccc P 108b8dd3313b43bb8db48abccbb8ddda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 11:35:32.693957 26966 log.cc:1196] T ab5067140d374c7cb605dd98198a9ccc P 108b8dd3313b43bb8db48abccbb8ddda: Deleting WAL directory at /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786361401922362-8267-0/minicluster-data/ts-2/wal/wals/ab5067140d374c7cb605dd98198a9ccc
I20260810 11:35:32.694723 26966 ts_tablet_manager.cc:1939] T ab5067140d374c7cb605dd98198a9ccc P 108b8dd3313b43bb8db48abccbb8ddda: Deleting consensus metadata
I20260810 11:35:32.697041 26449 catalog_manager.cc:4841] TS 108b8dd3313b43bb8db48abccbb8ddda (127.8.18.195:32913): tablet ab5067140d374c7cb605dd98198a9ccc (table test-table [id=21bd3243147640c7a5035a795f4a817b]) successfully deleted
I20260810 11:35:32.706262 26965 ts_tablet_manager.cc:1918] T ab5067140d374c7cb605dd98198a9ccc P d4af19f6c22946d095eba3410c57bcf7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 11:35:32.706637 26965 log.cc:1196] T ab5067140d374c7cb605dd98198a9ccc P d4af19f6c22946d095eba3410c57bcf7: Deleting WAL directory at /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786361401922362-8267-0/minicluster-data/ts-1/wal/wals/ab5067140d374c7cb605dd98198a9ccc
I20260810 11:35:32.707435 26965 ts_tablet_manager.cc:1939] T ab5067140d374c7cb605dd98198a9ccc P d4af19f6c22946d095eba3410c57bcf7: Deleting consensus metadata
I20260810 11:35:32.709684 26450 catalog_manager.cc:4841] TS d4af19f6c22946d095eba3410c57bcf7 (127.8.18.194:36019): tablet ab5067140d374c7cb605dd98198a9ccc (table test-table [id=21bd3243147640c7a5035a795f4a817b]) successfully deleted
I20260810 11:35:32.789921 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 26523
I20260810 11:35:32.812000 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 26657
W20260810 11:35:32.812011 26448 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.18.193:44019 (error 108)
I20260810 11:35:32.815650 26936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:32.815116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.193:44019 (local address 127.0.0.1:45700)
0810 11:35:32.815327 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0810 11:35:32.815459 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.193:44019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
W20260810 11:35:32.815975 26448 catalog_manager.cc:4584] TS df4402bb11b84a16a0609b3a2373dc79 (127.8.18.193:44019): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ab5067140d374c7cb605dd98198a9ccc: Network error: Client connection negotiation failed: client connection to 127.8.18.193:44019: connect: Connection refused (error 111)
I20260810 11:35:32.821516 26935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:32.813296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58834 (local address 127.8.18.194:36019)
0810 11:35:32.813530 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0810 11:35:32.813536 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:35:32.813559 (+ 23us) server_negotiation.cc:407] Connection header received
0810 11:35:32.813634 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:35:32.813639 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:35:32.813740 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:35:32.813850 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:35:32.814465 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:32.815535 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:32.817444 (+ 1909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:32.818177 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:32.818323 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:35:32.819678 (+ 1355us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:35:32.819691 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:35:32.819709 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:35:32.820534 (+ 825us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:35:32.820603 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:35:32.820635 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:35:32.820772 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0810 11:35:32.820943 (+ 171us) server_negotiation.cc:299] Negotiation successful
0810 11:35:32.821089 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260810 11:35:32.822427 26781 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:35:32.870183 26936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:32.869523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.193:44019 (local address 127.0.0.1:45702)
0810 11:35:32.869777 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0810 11:35:32.869894 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.193:44019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260810 11:35:32.871644 26936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:32.871167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.193:44019 (local address 127.0.0.1:45718)
0810 11:35:32.871354 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0810 11:35:32.871470 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.193:44019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260810 11:35:32.919436 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 26657
I20260810 11:35:32.932519 26936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:32.931845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.193:44019 (local address 127.0.0.1:45720)
0810 11:35:32.932140 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0810 11:35:32.932267 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.193:44019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108}
I20260810 11:35:32.934128 26936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:32.933596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.193:44019 (local address 127.0.0.1:45724)
0810 11:35:32.933803 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0810 11:35:32.933920 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.193:44019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260810 11:35:32.941025 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 26790
I20260810 11:35:32.950639 26938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:32.942007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54430 (local address 127.8.18.195:32913)
0810 11:35:32.942231 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0810 11:35:32.942237 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:35:32.942261 (+ 24us) server_negotiation.cc:407] Connection header received
0810 11:35:32.942461 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:35:32.942467 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:35:32.942614 (+ 147us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:35:32.942744 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:35:32.943206 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:32.944480 (+ 1274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:32.946312 (+ 1832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:32.947083 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:32.947260 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:35:32.948633 (+ 1373us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:35:32.948651 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:35:32.948683 (+ 32us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:35:32.949677 (+ 994us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:35:32.949748 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:35:32.949777 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:35:32.949899 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0810 11:35:32.950087 (+ 188us) server_negotiation.cc:299] Negotiation successful
0810 11:35:32.950226 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260810 11:35:32.951634 26914 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:35:33.015228 26936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:33.014567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.193:44019 (local address 127.0.0.1:45732)
0810 11:35:33.014854 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0810 11:35:33.014976 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.193:44019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119}
I20260810 11:35:33.016513 26936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:33.016021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.193:44019 (local address 127.0.0.1:45744)
0810 11:35:33.016271 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0810 11:35:33.016375 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.193:44019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118}
I20260810 11:35:33.080667 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 26790
I20260810 11:35:33.102736 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 26432
I20260810 11:35:33.103942 26494 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:35:33.218580 26936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:33.217643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.193:44019 (local address 127.0.0.1:45748)
0810 11:35:33.217997 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0810 11:35:33.218142 (+ 145us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.193:44019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191}
I20260810 11:35:33.218854 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 26432
2026-08-10T11:35:33Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (3122 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786361401922362-8267-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 11:35:33 dist-test-slave-srgv krb5kdc[26975](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 11:35:33 dist-test-slave-srgv krb5kdc[26975](info): set up 2 sockets
Aug 10 11:35:33 dist-test-slave-srgv krb5kdc[26975](info): commencing operation
krb5kdc: starting...
W20260810 11:35:34.385020 8267 mini_kdc.cc:121] Time spent starting KDC: real 1.119s user 0.006s sys 0.035s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786361401922362-8267-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786361401922362-8267-0/krb5kdc/test-user.keytab.
Aug 10 11:35:34 dist-test-slave-srgv krb5kdc[26975](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361734, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T11:35:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T11:35:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.18.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.18.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.18.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.254.keytab.
Entry for principal kudu/127.8.18.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.254.keytab.
I20260810 11:35:34.705590 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786361401922362-8267-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786361401922362-8267-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786361401922362-8267-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.18.254:34179
--webserver_interface=127.8.18.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.254.keytab
--principal=kudu/127.8.18.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.18.212:38345
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.18.254:34179
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786361401922362-8267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786361401922362-8267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786361401922362-8267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:35:35.068809 26988 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:35:35.069144 26988 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:35:35.069239 26988 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:35:35.076982 26988 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 11:35:35.077139 26988 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:35:35.077191 26988 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:35:35.077257 26988 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 11:35:35.077306 26988 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 11:35:35.088049 26988 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.18.212:38345
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786361401922362-8267-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786361401922362-8267-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.18.254:34179
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.254.keytab
--principal=kudu/127.8.18.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.18.254:34179
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.18.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786361401922362-8267-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:35:35.089608 26988 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:35:35.091043 26988 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:35:35.103871 26993 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:35:35.103966 26994 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:35:35.104041 26996 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:35:35.105273 26988 server_base.cc:1034] running on GCE node
I20260810 11:35:35.106249 26988 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:35:35.107946 26988 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:35:35.109241 26988 hybrid_clock.cc:648] HybridClock initialized: now 1786361735109183 us; error 75 us; skew 500 ppm
I20260810 11:35:35.109738 26988 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:35:35 dist-test-slave-srgv krb5kdc[26975](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361735, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:35:35.113322 26988 init.cc:377] Logged in from keytab as kudu/127.8.18.254@KRBTEST.COM (short username kudu)
I20260810 11:35:35.115334 26988 webserver.cc:460] Webserver started at http://127.8.18.254:36491/ using document root <none> and password file <none>
I20260810 11:35:35.116079 26988 fs_manager.cc:359] Metadata directory not provided
I20260810 11:35:35.116204 26988 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:35:35.116477 26988 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:35:35.119889 26988 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786361401922362-8267-0/minicluster-data/master-0/data/instance:
uuid: "8207377d1f6a4c9d8f07bd3cbd83ca78"
format_stamp: "Formatted at 2026-08-10 11:35:35 on dist-test-slave-srgv"
server_key: "91a596aea4c5ee3ea071a3ac57fd8020"
server_key_iv: "88d62cda525603db1e78ce617a4df2b7"
server_key_version: "encryptionkey@0"
I20260810 11:35:35.120725 26988 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786361401922362-8267-0/minicluster-data/master-0/wal/instance:
uuid: "8207377d1f6a4c9d8f07bd3cbd83ca78"
format_stamp: "Formatted at 2026-08-10 11:35:35 on dist-test-slave-srgv"
server_key: "91a596aea4c5ee3ea071a3ac57fd8020"
server_key_iv: "88d62cda525603db1e78ce617a4df2b7"
server_key_version: "encryptionkey@0"
I20260810 11:35:35.126641 26988 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.004s
I20260810 11:35:35.130806 27003 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:35:35.132601 26988 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260810 11:35:35.132867 26988 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786361401922362-8267-0/minicluster-data/master-0/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786361401922362-8267-0/minicluster-data/master-0/wal
uuid: "8207377d1f6a4c9d8f07bd3cbd83ca78"
format_stamp: "Formatted at 2026-08-10 11:35:35 on dist-test-slave-srgv"
server_key: "91a596aea4c5ee3ea071a3ac57fd8020"
server_key_iv: "88d62cda525603db1e78ce617a4df2b7"
server_key_version: "encryptionkey@0"
I20260810 11:35:35.133157 26988 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786361401922362-8267-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786361401922362-8267-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786361401922362-8267-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:35:35.150368 26988 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:35:35.155179 26988 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:35:35.155547 26988 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:35:35.177217 26988 rpc_server.cc:307] RPC server started. Bound to: 127.8.18.254:34179
I20260810 11:35:35.177254 27057 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.18.254:34179 every 8 connection(s)
I20260810 11:35:35.178850 26988 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
I20260810 11:35:35.183298 27058 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:35:35.188634 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 26988
I20260810 11:35:35.188855 8267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786361401922362-8267-0/minicluster-data/master-0/wal/instance
I20260810 11:35:35.189447 8267 external_mini_cluster.cc:1428] Setting key bb8fbc848eefc4148a5b89867dd7aa0a
I20260810 11:35:35.198751 27058 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8207377d1f6a4c9d8f07bd3cbd83ca78: Bootstrap starting.
I20260810 11:35:35.203280 27058 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8207377d1f6a4c9d8f07bd3cbd83ca78: Neither blocks nor log segments found. Creating new log.
I20260810 11:35:35.204619 27058 log.cc:824] T 00000000000000000000000000000000 P 8207377d1f6a4c9d8f07bd3cbd83ca78: Log is configured to *not* fsync() on all Append() calls
Aug 10 11:35:35 dist-test-slave-srgv krb5kdc[26975](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361734, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.18.254@KRBTEST.COM
I20260810 11:35:35.208673 27058 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8207377d1f6a4c9d8f07bd3cbd83ca78: No bootstrap required, opened a new log
I20260810 11:35:35.216396 27061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:35.191608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35880 (local address 127.8.18.254:34179)
0810 11:35:35.192515 (+ 907us) server_negotiation.cc:206] Beginning negotiation
0810 11:35:35.192528 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 11:35:35.192572 (+ 44us) server_negotiation.cc:407] Connection header received
0810 11:35:35.194475 (+ 1903us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:35:35.194505 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:35:35.194984 (+ 479us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:35:35.195427 (+ 443us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:35:35.197167 (+ 1740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:35.198647 (+ 1480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:35.200350 (+ 1703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:35.201308 (+ 958us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:35.201593 (+ 285us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:35:35.207485 (+ 5892us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:35:35.207495 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:35:35.207512 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:35:35.207556 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:35:35.211180 (+ 3624us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:35:35.212401 (+ 1221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:35:35.212410 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:35:35.212418 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:35:35.212647 (+ 229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:35:35.213166 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:35:35.213169 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:35:35.213171 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:35:35.213887 (+ 716us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:35:35.214138 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0810 11:35:35.214763 (+ 625us) server_negotiation.cc:299] Negotiation successful
0810 11:35:35.215144 (+ 381us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":674,"thread_start_us":425,"threads_started":1}
I20260810 11:35:35.219374 27058 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8207377d1f6a4c9d8f07bd3cbd83ca78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8207377d1f6a4c9d8f07bd3cbd83ca78" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 34179 } }
I20260810 11:35:35.219815 27058 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8207377d1f6a4c9d8f07bd3cbd83ca78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:35:35.219926 27058 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8207377d1f6a4c9d8f07bd3cbd83ca78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8207377d1f6a4c9d8f07bd3cbd83ca78, State: Initialized, Role: FOLLOWER
I20260810 11:35:35.220546 27058 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8207377d1f6a4c9d8f07bd3cbd83ca78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8207377d1f6a4c9d8f07bd3cbd83ca78" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 34179 } }
I20260810 11:35:35.220804 27058 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8207377d1f6a4c9d8f07bd3cbd83ca78 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 11:35:35.220948 27058 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8207377d1f6a4c9d8f07bd3cbd83ca78 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 11:35:35.221118 27058 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8207377d1f6a4c9d8f07bd3cbd83ca78 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:35:35.223592 27058 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8207377d1f6a4c9d8f07bd3cbd83ca78 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8207377d1f6a4c9d8f07bd3cbd83ca78" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 34179 } }
I20260810 11:35:35.224174 27058 leader_election.cc:302] T 00000000000000000000000000000000 P 8207377d1f6a4c9d8f07bd3cbd83ca78 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8207377d1f6a4c9d8f07bd3cbd83ca78; no voters:
I20260810 11:35:35.224833 27058 leader_election.cc:288] T 00000000000000000000000000000000 P 8207377d1f6a4c9d8f07bd3cbd83ca78 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 11:35:35.225008 27063 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8207377d1f6a4c9d8f07bd3cbd83ca78 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:35:35.226368 27063 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8207377d1f6a4c9d8f07bd3cbd83ca78 [term 1 LEADER]: Becoming Leader. State: Replica: 8207377d1f6a4c9d8f07bd3cbd83ca78, State: Running, Role: LEADER
I20260810 11:35:35.226998 27063 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8207377d1f6a4c9d8f07bd3cbd83ca78 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8207377d1f6a4c9d8f07bd3cbd83ca78" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 34179 } }
I20260810 11:35:35.227314 27058 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8207377d1f6a4c9d8f07bd3cbd83ca78 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 11:35:35.230151 27064 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8207377d1f6a4c9d8f07bd3cbd83ca78 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8207377d1f6a4c9d8f07bd3cbd83ca78" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8207377d1f6a4c9d8f07bd3cbd83ca78" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 34179 } } }
I20260810 11:35:35.230633 27064 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8207377d1f6a4c9d8f07bd3cbd83ca78 [sys.catalog]: This master's current role is: LEADER
I20260810 11:35:35.231312 27065 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8207377d1f6a4c9d8f07bd3cbd83ca78 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8207377d1f6a4c9d8f07bd3cbd83ca78. Latest consensus state: current_term: 1 leader_uuid: "8207377d1f6a4c9d8f07bd3cbd83ca78" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8207377d1f6a4c9d8f07bd3cbd83ca78" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 34179 } } }
I20260810 11:35:35.231695 27065 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8207377d1f6a4c9d8f07bd3cbd83ca78 [sys.catalog]: This master's current role is: LEADER
I20260810 11:35:35.232930 27071 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 11:35:35.238868 27071 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 11:35:35.251986 27071 catalog_manager.cc:1347] Generated new cluster ID: 0576bcb608c94e2cb4654a02c71e54c8
I20260810 11:35:35.252245 27071 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 11:35:35.277737 27071 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 11:35:35.279943 27071 catalog_manager.cc:1504] Loading token signing keys...
I20260810 11:35:35.290140 27071 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8207377d1f6a4c9d8f07bd3cbd83ca78: Generated new TSK 0
I20260810 11:35:35.291469 27071 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.18.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.18.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.18.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.193.keytab.
Entry for principal kudu/127.8.18.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.193.keytab.
I20260810 11:35:35.376466 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786361401922362-8267-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786361401922362-8267-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786361401922362-8267-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786361401922362-8267-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.18.193:0
--local_ip_for_outbound_sockets=127.8.18.193
--webserver_interface=127.8.18.193
--webserver_port=0
--tserver_master_addrs=127.8.18.254:34179
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.193.keytab
--principal=kudu/127.8.18.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.18.212:38345
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786361401922362-8267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786361401922362-8267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786361401922362-8267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:35:35.763576 27084 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:35:35.764055 27084 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:35:35.764231 27084 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:35:35.772151 27084 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:35:35.772302 27084 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:35:35.772428 27084 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.18.193
I20260810 11:35:35.783223 27084 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.18.212:38345
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786361401922362-8267-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786361401922362-8267-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.193.keytab
--principal=kudu/127.8.18.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.18.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786361401922362-8267-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.18.193
--webserver_port=0
--tserver_master_addrs=127.8.18.254:34179
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.18.193
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786361401922362-8267-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:35:35.784685 27084 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:35:35.786127 27084 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:35:35.801011 27089 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:35:35.800921 27092 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:35:35.800892 27090 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:35:35.802412 27084 server_base.cc:1034] running on GCE node
I20260810 11:35:35.803189 27084 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:35:35.804277 27084 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:35:35.805534 27084 hybrid_clock.cc:648] HybridClock initialized: now 1786361735805481 us; error 48 us; skew 500 ppm
I20260810 11:35:35.805913 27084 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:35:35 dist-test-slave-srgv krb5kdc[26975](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361735, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:35:35.813910 27084 init.cc:377] Logged in from keytab as kudu/127.8.18.193@KRBTEST.COM (short username kudu)
I20260810 11:35:35.815804 27084 webserver.cc:460] Webserver started at http://127.8.18.193:38695/ using document root <none> and password file <none>
I20260810 11:35:35.816619 27084 fs_manager.cc:359] Metadata directory not provided
I20260810 11:35:35.816793 27084 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:35:35.817170 27084 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:35:35.820554 27084 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786361401922362-8267-0/minicluster-data/ts-0/data/instance:
uuid: "1797c22511034100a7414a5afd670ae6"
format_stamp: "Formatted at 2026-08-10 11:35:35 on dist-test-slave-srgv"
server_key: "7bc35dd4eab24b934147d3910f4f7392"
server_key_iv: "d7565b679784cd5709ba0a41e1419e70"
server_key_version: "encryptionkey@0"
I20260810 11:35:35.821539 27084 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786361401922362-8267-0/minicluster-data/ts-0/wal/instance:
uuid: "1797c22511034100a7414a5afd670ae6"
format_stamp: "Formatted at 2026-08-10 11:35:35 on dist-test-slave-srgv"
server_key: "7bc35dd4eab24b934147d3910f4f7392"
server_key_iv: "d7565b679784cd5709ba0a41e1419e70"
server_key_version: "encryptionkey@0"
I20260810 11:35:35.828018 27084 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.001s
I20260810 11:35:35.832830 27099 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:35:35.834739 27084 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260810 11:35:35.835086 27084 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786361401922362-8267-0/minicluster-data/ts-0/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786361401922362-8267-0/minicluster-data/ts-0/wal
uuid: "1797c22511034100a7414a5afd670ae6"
format_stamp: "Formatted at 2026-08-10 11:35:35 on dist-test-slave-srgv"
server_key: "7bc35dd4eab24b934147d3910f4f7392"
server_key_iv: "d7565b679784cd5709ba0a41e1419e70"
server_key_version: "encryptionkey@0"
I20260810 11:35:35.835347 27084 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786361401922362-8267-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786361401922362-8267-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786361401922362-8267-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:35:35.856269 27084 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:35:35.861663 27084 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:35:35.862001 27084 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:35:35.863265 27084 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:35:35.864985 27084 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:35:35.865106 27084 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:35:35.865281 27084 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:35:35.865360 27084 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:35:35.904062 27084 rpc_server.cc:307] RPC server started. Bound to: 127.8.18.193:36015
I20260810 11:35:35.904172 27217 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.18.193:36015 every 8 connection(s)
I20260810 11:35:35.905483 27084 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786361401922362-8267-0/minicluster-data/ts-0/data/info.pb
I20260810 11:35:35.910457 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 27084
I20260810 11:35:35.910684 8267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786361401922362-8267-0/minicluster-data/ts-0/wal/instance
I20260810 11:35:35.911206 8267 external_mini_cluster.cc:1428] Setting key 51e977fec09861b96b6df9bb256559b8
Aug 10 11:35:35 dist-test-slave-srgv krb5kdc[26975](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361735, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.193@KRBTEST.COM for kudu/127.8.18.254@KRBTEST.COM
I20260810 11:35:35.930832 27220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:35.908262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.193:46297 (local address 127.8.18.254:34179)
0810 11:35:35.908771 (+ 509us) server_negotiation.cc:206] Beginning negotiation
0810 11:35:35.908776 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:35:35.909923 (+ 1147us) server_negotiation.cc:407] Connection header received
0810 11:35:35.912196 (+ 2273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:35:35.912202 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:35:35.912307 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:35:35.912442 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:35:35.914710 (+ 2268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:35.915959 (+ 1249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:35.917887 (+ 1928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:35.918652 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:35.918821 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:35:35.923045 (+ 4224us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:35:35.923056 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:35:35.923064 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:35:35.923121 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:35:35.926495 (+ 3374us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:35:35.927923 (+ 1428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:35:35.927928 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:35:35.927930 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:35:35.928067 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:35:35.928693 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:35:35.928700 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:35:35.928703 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:35:35.929008 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:35:35.929127 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0810 11:35:35.930245 (+ 1118us) server_negotiation.cc:299] Negotiation successful
0810 11:35:35.930447 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":385,"thread_start_us":276,"threads_started":1}
I20260810 11:35:35.932384 27221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:35.908763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:34179 (local address 127.8.18.193:46297)
0810 11:35:35.909681 (+ 918us) negotiation.cc:107] Waiting for socket to connect
0810 11:35:35.909744 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0810 11:35:35.911823 (+ 2079us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:35:35.912797 (+ 974us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:35:35.912812 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:35:35.913809 (+ 997us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:35:35.914484 (+ 675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:35:35.914505 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:35:35.916212 (+ 1707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:35:35.916216 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:35:35.917692 (+ 1476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:35:35.917709 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:35:35.918924 (+ 1215us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:35:35.918929 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:35:35.919170 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:35:35.920013 (+ 843us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:35:35.920046 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:35:35.922669 (+ 2623us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:35:35.926708 (+ 4039us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:35:35.926722 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:35:35.926744 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:35:35.927761 (+ 1017us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:35:35.928192 (+ 431us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:35:35.928200 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:35:35.928207 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:35:35.928501 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:35:35.929154 (+ 653us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:35:35.929166 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:35:35.929771 (+ 605us) client_negotiation.cc:769] Sending connection context
0810 11:35:35.930114 (+ 343us) client_negotiation.cc:240] Negotiation successful
0810 11:35:35.930784 (+ 670us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":565,"thread_start_us":362,"threads_started":1}
I20260810 11:35:35.934278 27218 heartbeater.cc:347] Connected to a master server at 127.8.18.254:34179
I20260810 11:35:35.934643 27218 heartbeater.cc:464] Registering TS with master...
I20260810 11:35:35.935676 27218 heartbeater.cc:511] Master 127.8.18.254:34179 requested a full tablet report, sending...
I20260810 11:35:35.938555 27020 ts_manager.cc:194] Registered new tserver with Master: 1797c22511034100a7414a5afd670ae6 (127.8.18.193:36015)
I20260810 11:35:35.940938 27020 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.18.193@KRBTEST.COM'} at 127.8.18.193:46297
WARNING: no policy specified for kudu/127.8.18.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.18.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.18.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.194.keytab.
Entry for principal kudu/127.8.18.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.194.keytab.
I20260810 11:35:35.993557 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786361401922362-8267-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786361401922362-8267-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786361401922362-8267-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786361401922362-8267-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.18.194:0
--local_ip_for_outbound_sockets=127.8.18.194
--webserver_interface=127.8.18.194
--webserver_port=0
--tserver_master_addrs=127.8.18.254:34179
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.194.keytab
--principal=kudu/127.8.18.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.18.212:38345
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786361401922362-8267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786361401922362-8267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786361401922362-8267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:35:36.370519 27224 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:35:36.370841 27224 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:35:36.371017 27224 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:35:36.378839 27224 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:35:36.378986 27224 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:35:36.379182 27224 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.18.194
I20260810 11:35:36.389737 27224 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.18.212:38345
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786361401922362-8267-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786361401922362-8267-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.194.keytab
--principal=kudu/127.8.18.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.18.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786361401922362-8267-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.18.194
--webserver_port=0
--tserver_master_addrs=127.8.18.254:34179
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.18.194
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786361401922362-8267-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:35:36.391222 27224 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:35:36.392655 27224 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:35:36.408236 27229 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:35:36.408167 27232 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:35:36.408120 27230 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:35:36.409463 27224 server_base.cc:1034] running on GCE node
I20260810 11:35:36.410383 27224 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:35:36.411907 27224 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:35:36.413223 27224 hybrid_clock.cc:648] HybridClock initialized: now 1786361736413131 us; error 59 us; skew 500 ppm
I20260810 11:35:36.413654 27224 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:35:36 dist-test-slave-srgv krb5kdc[26975](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361736, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:35:36.417279 27224 init.cc:377] Logged in from keytab as kudu/127.8.18.194@KRBTEST.COM (short username kudu)
I20260810 11:35:36.419083 27224 webserver.cc:460] Webserver started at http://127.8.18.194:40637/ using document root <none> and password file <none>
I20260810 11:35:36.419852 27224 fs_manager.cc:359] Metadata directory not provided
I20260810 11:35:36.420017 27224 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:35:36.420377 27224 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:35:36.423666 27224 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786361401922362-8267-0/minicluster-data/ts-1/data/instance:
uuid: "78a178dd3aa744e3abbf551a85de44bf"
format_stamp: "Formatted at 2026-08-10 11:35:36 on dist-test-slave-srgv"
server_key: "b60c2e006fc7f95629af34e81b32c8ea"
server_key_iv: "7319274ce14f45d6753aceebdcde08a6"
server_key_version: "encryptionkey@0"
I20260810 11:35:36.424582 27224 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786361401922362-8267-0/minicluster-data/ts-1/wal/instance:
uuid: "78a178dd3aa744e3abbf551a85de44bf"
format_stamp: "Formatted at 2026-08-10 11:35:36 on dist-test-slave-srgv"
server_key: "b60c2e006fc7f95629af34e81b32c8ea"
server_key_iv: "7319274ce14f45d6753aceebdcde08a6"
server_key_version: "encryptionkey@0"
I20260810 11:35:36.430742 27224 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260810 11:35:36.434901 27239 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:35:36.436510 27224 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 11:35:36.436761 27224 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786361401922362-8267-0/minicluster-data/ts-1/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786361401922362-8267-0/minicluster-data/ts-1/wal
uuid: "78a178dd3aa744e3abbf551a85de44bf"
format_stamp: "Formatted at 2026-08-10 11:35:36 on dist-test-slave-srgv"
server_key: "b60c2e006fc7f95629af34e81b32c8ea"
server_key_iv: "7319274ce14f45d6753aceebdcde08a6"
server_key_version: "encryptionkey@0"
I20260810 11:35:36.437037 27224 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786361401922362-8267-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786361401922362-8267-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786361401922362-8267-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:35:36.459033 27224 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:35:36.464242 27224 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:35:36.464655 27224 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:35:36.465910 27224 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:35:36.467765 27224 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:35:36.467864 27224 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:35:36.468010 27224 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:35:36.468065 27224 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:35:36.505908 27224 rpc_server.cc:307] RPC server started. Bound to: 127.8.18.194:42405
I20260810 11:35:36.506007 27354 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.18.194:42405 every 8 connection(s)
I20260810 11:35:36.507351 27224 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786361401922362-8267-0/minicluster-data/ts-1/data/info.pb
I20260810 11:35:36.508515 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 27224
I20260810 11:35:36.508728 8267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786361401922362-8267-0/minicluster-data/ts-1/wal/instance
I20260810 11:35:36.509213 8267 external_mini_cluster.cc:1428] Setting key 9c26042a45edd37c03851ec23118e2c0
Aug 10 11:35:36 dist-test-slave-srgv krb5kdc[26975](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361736, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.194@KRBTEST.COM for kudu/127.8.18.254@KRBTEST.COM
I20260810 11:35:36.533597 27357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:36.510315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.194:49641 (local address 127.8.18.254:34179)
0810 11:35:36.510850 (+ 535us) server_negotiation.cc:206] Beginning negotiation
0810 11:35:36.510856 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:35:36.512086 (+ 1230us) server_negotiation.cc:407] Connection header received
0810 11:35:36.515297 (+ 3211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:35:36.515302 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:35:36.515382 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:35:36.515492 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:35:36.517963 (+ 2471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:36.518736 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:36.521509 (+ 2773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:36.521985 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:36.522101 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:35:36.526760 (+ 4659us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:35:36.526765 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:35:36.526769 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:35:36.526809 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:35:36.529891 (+ 3082us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:35:36.530795 (+ 904us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:35:36.530801 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:35:36.530803 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:35:36.530949 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:35:36.531461 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:35:36.531466 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:35:36.531468 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:35:36.531778 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:35:36.531958 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0810 11:35:36.532847 (+ 889us) server_negotiation.cc:299] Negotiation successful
0810 11:35:36.533146 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":402,"thread_start_us":268,"threads_started":1}
I20260810 11:35:36.534670 27358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:36.510980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:34179 (local address 127.8.18.194:49641)
0810 11:35:36.511874 (+ 894us) negotiation.cc:107] Waiting for socket to connect
0810 11:35:36.511939 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0810 11:35:36.514887 (+ 2948us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:35:36.515826 (+ 939us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:35:36.515841 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:35:36.516754 (+ 913us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:35:36.517702 (+ 948us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:35:36.517734 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:35:36.518930 (+ 1196us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:35:36.518939 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:35:36.521246 (+ 2307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:35:36.521267 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:35:36.522278 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:35:36.522288 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:35:36.522653 (+ 365us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:35:36.523472 (+ 819us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:35:36.523515 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:35:36.526441 (+ 2926us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:35:36.530078 (+ 3637us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:35:36.530088 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:35:36.530109 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:35:36.530626 (+ 517us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:35:36.531077 (+ 451us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:35:36.531083 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:35:36.531086 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:35:36.531313 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:35:36.531925 (+ 612us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:35:36.531934 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:35:36.532394 (+ 460us) client_negotiation.cc:769] Sending connection context
0810 11:35:36.532736 (+ 342us) client_negotiation.cc:240] Negotiation successful
0810 11:35:36.533366 (+ 630us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":580,"thread_start_us":354,"threads_started":1}
I20260810 11:35:36.536580 27355 heartbeater.cc:347] Connected to a master server at 127.8.18.254:34179
I20260810 11:35:36.537061 27355 heartbeater.cc:464] Registering TS with master...
I20260810 11:35:36.538215 27355 heartbeater.cc:511] Master 127.8.18.254:34179 requested a full tablet report, sending...
I20260810 11:35:36.540385 27020 ts_manager.cc:194] Registered new tserver with Master: 78a178dd3aa744e3abbf551a85de44bf (127.8.18.194:42405)
I20260810 11:35:36.541561 27020 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.18.194@KRBTEST.COM'} at 127.8.18.194:49641
WARNING: no policy specified for kudu/127.8.18.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.18.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.18.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.195.keytab.
Entry for principal kudu/127.8.18.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.195.keytab.
I20260810 11:35:36.583372 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786361401922362-8267-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786361401922362-8267-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786361401922362-8267-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786361401922362-8267-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.18.195:0
--local_ip_for_outbound_sockets=127.8.18.195
--webserver_interface=127.8.18.195
--webserver_port=0
--tserver_master_addrs=127.8.18.254:34179
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.195.keytab
--principal=kudu/127.8.18.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.18.212:38345
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786361401922362-8267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786361401922362-8267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786361401922362-8267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 11:35:36.945712 27218 heartbeater.cc:503] Master 127.8.18.254:34179 was elected leader, sending a full tablet report...
W20260810 11:35:36.961040 27361 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:35:36.961462 27361 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:35:36.961649 27361 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:35:36.970168 27361 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:35:36.970364 27361 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:35:36.970566 27361 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.18.195
I20260810 11:35:36.981630 27361 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.18.212:38345
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786361401922362-8267-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786361401922362-8267-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.195.keytab
--principal=kudu/127.8.18.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.18.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786361401922362-8267-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.18.195
--webserver_port=0
--tserver_master_addrs=127.8.18.254:34179
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.18.195
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786361401922362-8267-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:35:36.983158 27361 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:35:36.984614 27361 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:35:36.999312 27367 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:35:36.999478 27366 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:35:36.999605 27369 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:35:37.000803 27361 server_base.cc:1034] running on GCE node
I20260810 11:35:37.001626 27361 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:35:37.002700 27361 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:35:37.003952 27361 hybrid_clock.cc:648] HybridClock initialized: now 1786361737003856 us; error 84 us; skew 500 ppm
I20260810 11:35:37.004423 27361 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:35:37 dist-test-slave-srgv krb5kdc[26975](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361737, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:35:37.008267 27361 init.cc:377] Logged in from keytab as kudu/127.8.18.195@KRBTEST.COM (short username kudu)
I20260810 11:35:37.010251 27361 webserver.cc:460] Webserver started at http://127.8.18.195:34771/ using document root <none> and password file <none>
I20260810 11:35:37.011053 27361 fs_manager.cc:359] Metadata directory not provided
I20260810 11:35:37.011232 27361 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:35:37.011595 27361 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:35:37.014940 27361 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786361401922362-8267-0/minicluster-data/ts-2/data/instance:
uuid: "312a2553119d4f37b3d724e39d3cdaff"
format_stamp: "Formatted at 2026-08-10 11:35:37 on dist-test-slave-srgv"
server_key: "b253004f1f10c59f5e80b4470233c549"
server_key_iv: "ababd0e381fa5f378258e6cff7e3b3d3"
server_key_version: "encryptionkey@0"
I20260810 11:35:37.015892 27361 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786361401922362-8267-0/minicluster-data/ts-2/wal/instance:
uuid: "312a2553119d4f37b3d724e39d3cdaff"
format_stamp: "Formatted at 2026-08-10 11:35:37 on dist-test-slave-srgv"
server_key: "b253004f1f10c59f5e80b4470233c549"
server_key_iv: "ababd0e381fa5f378258e6cff7e3b3d3"
server_key_version: "encryptionkey@0"
I20260810 11:35:37.022133 27361 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260810 11:35:37.026460 27376 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:35:37.028133 27361 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260810 11:35:37.028367 27361 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786361401922362-8267-0/minicluster-data/ts-2/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786361401922362-8267-0/minicluster-data/ts-2/wal
uuid: "312a2553119d4f37b3d724e39d3cdaff"
format_stamp: "Formatted at 2026-08-10 11:35:37 on dist-test-slave-srgv"
server_key: "b253004f1f10c59f5e80b4470233c549"
server_key_iv: "ababd0e381fa5f378258e6cff7e3b3d3"
server_key_version: "encryptionkey@0"
I20260810 11:35:37.028614 27361 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786361401922362-8267-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786361401922362-8267-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786361401922362-8267-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:35:37.056468 27361 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:35:37.061754 27361 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:35:37.062127 27361 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:35:37.063262 27361 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:35:37.064913 27361 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:35:37.065042 27361 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:35:37.065214 27361 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:35:37.065289 27361 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:35:37.102928 27361 rpc_server.cc:307] RPC server started. Bound to: 127.8.18.195:40593
I20260810 11:35:37.102948 27491 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.18.195:40593 every 8 connection(s)
I20260810 11:35:37.104372 27361 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786361401922362-8267-0/minicluster-data/ts-2/data/info.pb
I20260810 11:35:37.107765 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 27361
I20260810 11:35:37.107971 8267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786361401922362-8267-0/minicluster-data/ts-2/wal/instance
I20260810 11:35:37.108456 8267 external_mini_cluster.cc:1428] Setting key 98792a65353aefb574aa9e6d2819ef63
Aug 10 11:35:37 dist-test-slave-srgv krb5kdc[26975](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361737, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.195@KRBTEST.COM for kudu/127.8.18.254@KRBTEST.COM
I20260810 11:35:37.126763 27494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:37.107439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.195:38493 (local address 127.8.18.254:34179)
0810 11:35:37.108015 (+ 576us) server_negotiation.cc:206] Beginning negotiation
0810 11:35:37.108022 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:35:37.109491 (+ 1469us) server_negotiation.cc:407] Connection header received
0810 11:35:37.111266 (+ 1775us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:35:37.111272 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:35:37.111383 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:35:37.111510 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:35:37.113163 (+ 1653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:37.114047 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:37.115723 (+ 1676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:37.116162 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:37.116312 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:35:37.120172 (+ 3860us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:35:37.120178 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:35:37.120181 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:35:37.120218 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:35:37.123288 (+ 3070us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:35:37.124140 (+ 852us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:35:37.124149 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:35:37.124154 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:35:37.124293 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:35:37.124764 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:35:37.124772 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:35:37.124777 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:35:37.125070 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:35:37.125258 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0810 11:35:37.126095 (+ 837us) server_negotiation.cc:299] Negotiation successful
0810 11:35:37.126346 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":412,"thread_start_us":254,"threads_started":1}
I20260810 11:35:37.127851 27495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:37.108169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:34179 (local address 127.8.18.195:38493)
0810 11:35:37.109052 (+ 883us) negotiation.cc:107] Waiting for socket to connect
0810 11:35:37.109100 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0810 11:35:37.110946 (+ 1846us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:35:37.111767 (+ 821us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:35:37.111776 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:35:37.112359 (+ 583us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:35:37.112970 (+ 611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:35:37.112989 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:35:37.114247 (+ 1258us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:35:37.114251 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:35:37.115550 (+ 1299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:35:37.115564 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:35:37.116419 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:35:37.116426 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:35:37.116684 (+ 258us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:35:37.117387 (+ 703us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:35:37.117460 (+ 73us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:35:37.119899 (+ 2439us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:35:37.123469 (+ 3570us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:35:37.123479 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:35:37.123499 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:35:37.123992 (+ 493us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:35:37.124417 (+ 425us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:35:37.124422 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:35:37.124425 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:35:37.124643 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:35:37.125222 (+ 579us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:35:37.125231 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:35:37.125683 (+ 452us) client_negotiation.cc:769] Sending connection context
0810 11:35:37.125998 (+ 315us) client_negotiation.cc:240] Negotiation successful
0810 11:35:37.126600 (+ 602us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":602,"thread_start_us":372,"threads_started":1}
I20260810 11:35:37.129501 27492 heartbeater.cc:347] Connected to a master server at 127.8.18.254:34179
I20260810 11:35:37.129863 27492 heartbeater.cc:464] Registering TS with master...
I20260810 11:35:37.130769 27492 heartbeater.cc:511] Master 127.8.18.254:34179 requested a full tablet report, sending...
I20260810 11:35:37.132494 27020 ts_manager.cc:194] Registered new tserver with Master: 312a2553119d4f37b3d724e39d3cdaff (127.8.18.195:40593)
I20260810 11:35:37.133551 27020 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.18.195@KRBTEST.COM'} at 127.8.18.195:38493
I20260810 11:35:37.137540 8267 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 11:35:37.159051 27494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:37.142628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48638 (local address 127.8.18.254:34179)
0810 11:35:37.142885 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0810 11:35:37.142895 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 11:35:37.143328 (+ 433us) server_negotiation.cc:407] Connection header received
0810 11:35:37.143569 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:35:37.143574 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:35:37.143654 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:35:37.143774 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:35:37.145351 (+ 1577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:37.146237 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:37.147769 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:37.148225 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:37.148385 (+ 160us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:35:37.151669 (+ 3284us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:35:37.151675 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:35:37.151679 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:35:37.151723 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:35:37.155050 (+ 3327us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:35:37.156315 (+ 1265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:35:37.156320 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:35:37.156323 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:35:37.156481 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:35:37.157120 (+ 639us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:35:37.157125 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:35:37.157127 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:35:37.157600 (+ 473us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:35:37.157925 (+ 325us) server_negotiation.cc:1035] Waiting for connection context
0810 11:35:37.158458 (+ 533us) server_negotiation.cc:299] Negotiation successful
0810 11:35:37.158705 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260810 11:35:37.164311 27020 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48638:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 11:35:37.171916 27020 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 11:35:37.215605 27505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:37.187378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.194:42405 (local address 127.0.0.1:57760)
0810 11:35:37.205848 (+ 18470us) negotiation.cc:107] Waiting for socket to connect
0810 11:35:37.205869 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 11:35:37.206021 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:35:37.206829 (+ 808us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:35:37.206834 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:35:37.206862 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:35:37.207490 (+ 628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:35:37.207506 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:35:37.209790 (+ 2284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:35:37.209800 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:35:37.212327 (+ 2527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:35:37.212341 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:35:37.214431 (+ 2090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:35:37.214438 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:35:37.214696 (+ 258us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:35:37.214765 (+ 69us) client_negotiation.cc:769] Sending connection context
0810 11:35:37.214915 (+ 150us) client_negotiation.cc:240] Negotiation successful
0810 11:35:37.215147 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18326,"mutex_wait_us":70,"thread_start_us":14447,"threads_started":1}
I20260810 11:35:37.215626 27509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:37.188118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57760 (local address 127.8.18.194:42405)
0810 11:35:37.199252 (+ 11134us) server_negotiation.cc:206] Beginning negotiation
0810 11:35:37.199268 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0810 11:35:37.206021 (+ 6753us) server_negotiation.cc:407] Connection header received
0810 11:35:37.206280 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:35:37.206289 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:35:37.206608 (+ 319us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:35:37.206840 (+ 232us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:35:37.207731 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:37.209356 (+ 1625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:37.212573 (+ 3217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:37.214149 (+ 1576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:37.214334 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:35:37.214601 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0810 11:35:37.215004 (+ 403us) server_negotiation.cc:299] Negotiation successful
0810 11:35:37.215237 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10821,"thread_start_us":10645,"threads_started":1}
I20260810 11:35:37.217162 27506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:37.186869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.195:40593 (local address 127.0.0.1:32846)
0810 11:35:37.202246 (+ 15377us) negotiation.cc:107] Waiting for socket to connect
0810 11:35:37.202295 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0810 11:35:37.202571 (+ 276us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:35:37.203674 (+ 1103us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:35:37.203683 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:35:37.203717 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:35:37.204368 (+ 651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:35:37.204386 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:35:37.209339 (+ 4953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:35:37.209344 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:35:37.213916 (+ 4572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:35:37.213933 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:35:37.216338 (+ 2405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:35:37.216346 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:35:37.216519 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:35:37.216554 (+ 35us) client_negotiation.cc:769] Sending connection context
0810 11:35:37.216658 (+ 104us) client_negotiation.cc:240] Negotiation successful
0810 11:35:37.216808 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15079,"thread_start_us":18588,"threads_started":1}
I20260810 11:35:37.217356 27507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:37.187946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32846 (local address 127.8.18.195:40593)
0810 11:35:37.201368 (+ 13422us) server_negotiation.cc:206] Beginning negotiation
0810 11:35:37.201377 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 11:35:37.202411 (+ 1034us) server_negotiation.cc:407] Connection header received
0810 11:35:37.202826 (+ 415us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:35:37.202835 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:35:37.203116 (+ 281us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:35:37.203275 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:35:37.207311 (+ 4036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:37.209133 (+ 1822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:37.214125 (+ 4992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:37.215742 (+ 1617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:37.215901 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:35:37.216152 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0810 11:35:37.216762 (+ 610us) server_negotiation.cc:299] Negotiation successful
0810 11:35:37.216959 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13218,"thread_start_us":13501,"threads_started":1}
I20260810 11:35:37.219199 27511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:37.204361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.193:36015 (local address 127.0.0.1:40756)
0810 11:35:37.210569 (+ 6208us) negotiation.cc:107] Waiting for socket to connect
0810 11:35:37.210586 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 11:35:37.210767 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:35:37.211405 (+ 638us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:35:37.211410 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:35:37.211440 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:35:37.211894 (+ 454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:35:37.211905 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:35:37.213981 (+ 2076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:35:37.213988 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:35:37.216392 (+ 2404us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:35:37.216409 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:35:37.218375 (+ 1966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:35:37.218381 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:35:37.218562 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:35:37.218597 (+ 35us) client_negotiation.cc:769] Sending connection context
0810 11:35:37.218711 (+ 114us) client_negotiation.cc:240] Negotiation successful
0810 11:35:37.218861 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6078,"thread_start_us":5864,"threads_started":1}
I20260810 11:35:37.219517 27510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:37.203427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40756 (local address 127.8.18.193:36015)
0810 11:35:37.205337 (+ 1910us) server_negotiation.cc:206] Beginning negotiation
0810 11:35:37.205347 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 11:35:37.210736 (+ 5389us) server_negotiation.cc:407] Connection header received
0810 11:35:37.210975 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:35:37.210982 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:35:37.211201 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:35:37.211422 (+ 221us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:35:37.212086 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:37.213803 (+ 1717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:37.216596 (+ 2793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:37.218152 (+ 1556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:37.218324 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:35:37.218582 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0810 11:35:37.218831 (+ 249us) server_negotiation.cc:299] Negotiation successful
0810 11:35:37.219017 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1700,"thread_start_us":1489,"threads_started":1}
I20260810 11:35:37.221127 27288 tablet_service.cc:1467] Processing CreateTablet for tablet 2bd59bb23f284886ba9ef7883f361ab3 (DEFAULT_TABLE table=test-table [id=59b89d15cce14636979cb091456adf49]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:35:37.222159 27424 tablet_service.cc:1467] Processing CreateTablet for tablet 2bd59bb23f284886ba9ef7883f361ab3 (DEFAULT_TABLE table=test-table [id=59b89d15cce14636979cb091456adf49]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:35:37.222851 27288 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2bd59bb23f284886ba9ef7883f361ab3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:35:37.223213 27150 tablet_service.cc:1467] Processing CreateTablet for tablet 2bd59bb23f284886ba9ef7883f361ab3 (DEFAULT_TABLE table=test-table [id=59b89d15cce14636979cb091456adf49]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:35:37.223734 27424 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2bd59bb23f284886ba9ef7883f361ab3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:35:37.224459 27150 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2bd59bb23f284886ba9ef7883f361ab3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:35:37.238387 27512 tablet_bootstrap.cc:492] T 2bd59bb23f284886ba9ef7883f361ab3 P 1797c22511034100a7414a5afd670ae6: Bootstrap starting.
I20260810 11:35:37.243039 27512 tablet_bootstrap.cc:654] T 2bd59bb23f284886ba9ef7883f361ab3 P 1797c22511034100a7414a5afd670ae6: Neither blocks nor log segments found. Creating new log.
I20260810 11:35:37.244424 27512 log.cc:824] T 2bd59bb23f284886ba9ef7883f361ab3 P 1797c22511034100a7414a5afd670ae6: Log is configured to *not* fsync() on all Append() calls
I20260810 11:35:37.245824 27513 tablet_bootstrap.cc:492] T 2bd59bb23f284886ba9ef7883f361ab3 P 78a178dd3aa744e3abbf551a85de44bf: Bootstrap starting.
I20260810 11:35:37.251731 27514 tablet_bootstrap.cc:492] T 2bd59bb23f284886ba9ef7883f361ab3 P 312a2553119d4f37b3d724e39d3cdaff: Bootstrap starting.
I20260810 11:35:37.253993 27513 tablet_bootstrap.cc:654] T 2bd59bb23f284886ba9ef7883f361ab3 P 78a178dd3aa744e3abbf551a85de44bf: Neither blocks nor log segments found. Creating new log.
I20260810 11:35:37.255483 27513 log.cc:824] T 2bd59bb23f284886ba9ef7883f361ab3 P 78a178dd3aa744e3abbf551a85de44bf: Log is configured to *not* fsync() on all Append() calls
I20260810 11:35:37.255569 27514 tablet_bootstrap.cc:654] T 2bd59bb23f284886ba9ef7883f361ab3 P 312a2553119d4f37b3d724e39d3cdaff: Neither blocks nor log segments found. Creating new log.
I20260810 11:35:37.256981 27514 log.cc:824] T 2bd59bb23f284886ba9ef7883f361ab3 P 312a2553119d4f37b3d724e39d3cdaff: Log is configured to *not* fsync() on all Append() calls
I20260810 11:35:37.257325 27512 tablet_bootstrap.cc:492] T 2bd59bb23f284886ba9ef7883f361ab3 P 1797c22511034100a7414a5afd670ae6: No bootstrap required, opened a new log
I20260810 11:35:37.257808 27512 ts_tablet_manager.cc:1397] T 2bd59bb23f284886ba9ef7883f361ab3 P 1797c22511034100a7414a5afd670ae6: Time spent bootstrapping tablet: real 0.020s user 0.007s sys 0.011s
I20260810 11:35:37.260417 27513 tablet_bootstrap.cc:492] T 2bd59bb23f284886ba9ef7883f361ab3 P 78a178dd3aa744e3abbf551a85de44bf: No bootstrap required, opened a new log
I20260810 11:35:37.260644 27514 tablet_bootstrap.cc:492] T 2bd59bb23f284886ba9ef7883f361ab3 P 312a2553119d4f37b3d724e39d3cdaff: No bootstrap required, opened a new log
I20260810 11:35:37.260793 27513 ts_tablet_manager.cc:1397] T 2bd59bb23f284886ba9ef7883f361ab3 P 78a178dd3aa744e3abbf551a85de44bf: Time spent bootstrapping tablet: real 0.015s user 0.010s sys 0.000s
I20260810 11:35:37.260941 27514 ts_tablet_manager.cc:1397] T 2bd59bb23f284886ba9ef7883f361ab3 P 312a2553119d4f37b3d724e39d3cdaff: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260810 11:35:37.269127 27513 raft_consensus.cc:348] T 2bd59bb23f284886ba9ef7883f361ab3 P 78a178dd3aa744e3abbf551a85de44bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "312a2553119d4f37b3d724e39d3cdaff" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 40593 } } peers { permanent_uuid: "78a178dd3aa744e3abbf551a85de44bf" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 42405 } } peers { permanent_uuid: "1797c22511034100a7414a5afd670ae6" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 36015 } }
I20260810 11:35:37.269691 27513 raft_consensus.cc:374] T 2bd59bb23f284886ba9ef7883f361ab3 P 78a178dd3aa744e3abbf551a85de44bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:35:37.269797 27513 raft_consensus.cc:729] T 2bd59bb23f284886ba9ef7883f361ab3 P 78a178dd3aa744e3abbf551a85de44bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 78a178dd3aa744e3abbf551a85de44bf, State: Initialized, Role: FOLLOWER
I20260810 11:35:37.269594 27512 raft_consensus.cc:348] T 2bd59bb23f284886ba9ef7883f361ab3 P 1797c22511034100a7414a5afd670ae6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "312a2553119d4f37b3d724e39d3cdaff" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 40593 } } peers { permanent_uuid: "78a178dd3aa744e3abbf551a85de44bf" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 42405 } } peers { permanent_uuid: "1797c22511034100a7414a5afd670ae6" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 36015 } }
I20260810 11:35:37.270107 27512 raft_consensus.cc:374] T 2bd59bb23f284886ba9ef7883f361ab3 P 1797c22511034100a7414a5afd670ae6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:35:37.270215 27512 raft_consensus.cc:729] T 2bd59bb23f284886ba9ef7883f361ab3 P 1797c22511034100a7414a5afd670ae6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1797c22511034100a7414a5afd670ae6, State: Initialized, Role: FOLLOWER
I20260810 11:35:37.270673 27513 consensus_queue.cc:260] T 2bd59bb23f284886ba9ef7883f361ab3 P 78a178dd3aa744e3abbf551a85de44bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "312a2553119d4f37b3d724e39d3cdaff" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 40593 } } peers { permanent_uuid: "78a178dd3aa744e3abbf551a85de44bf" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 42405 } } peers { permanent_uuid: "1797c22511034100a7414a5afd670ae6" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 36015 } }
I20260810 11:35:37.270866 27512 consensus_queue.cc:260] T 2bd59bb23f284886ba9ef7883f361ab3 P 1797c22511034100a7414a5afd670ae6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "312a2553119d4f37b3d724e39d3cdaff" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 40593 } } peers { permanent_uuid: "78a178dd3aa744e3abbf551a85de44bf" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 42405 } } peers { permanent_uuid: "1797c22511034100a7414a5afd670ae6" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 36015 } }
I20260810 11:35:37.272020 27355 heartbeater.cc:503] Master 127.8.18.254:34179 was elected leader, sending a full tablet report...
I20260810 11:35:37.272887 27514 raft_consensus.cc:348] T 2bd59bb23f284886ba9ef7883f361ab3 P 312a2553119d4f37b3d724e39d3cdaff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "312a2553119d4f37b3d724e39d3cdaff" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 40593 } } peers { permanent_uuid: "78a178dd3aa744e3abbf551a85de44bf" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 42405 } } peers { permanent_uuid: "1797c22511034100a7414a5afd670ae6" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 36015 } }
I20260810 11:35:37.273545 27514 raft_consensus.cc:374] T 2bd59bb23f284886ba9ef7883f361ab3 P 312a2553119d4f37b3d724e39d3cdaff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:35:37.273672 27514 raft_consensus.cc:729] T 2bd59bb23f284886ba9ef7883f361ab3 P 312a2553119d4f37b3d724e39d3cdaff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 312a2553119d4f37b3d724e39d3cdaff, State: Initialized, Role: FOLLOWER
I20260810 11:35:37.274439 27514 consensus_queue.cc:260] T 2bd59bb23f284886ba9ef7883f361ab3 P 312a2553119d4f37b3d724e39d3cdaff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "312a2553119d4f37b3d724e39d3cdaff" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 40593 } } peers { permanent_uuid: "78a178dd3aa744e3abbf551a85de44bf" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 42405 } } peers { permanent_uuid: "1797c22511034100a7414a5afd670ae6" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 36015 } }
I20260810 11:35:37.274761 27513 ts_tablet_manager.cc:1428] T 2bd59bb23f284886ba9ef7883f361ab3 P 78a178dd3aa744e3abbf551a85de44bf: Time spent starting tablet: real 0.014s user 0.013s sys 0.000s
I20260810 11:35:37.276634 27492 heartbeater.cc:503] Master 127.8.18.254:34179 was elected leader, sending a full tablet report...
I20260810 11:35:37.276928 27514 ts_tablet_manager.cc:1428] T 2bd59bb23f284886ba9ef7883f361ab3 P 312a2553119d4f37b3d724e39d3cdaff: Time spent starting tablet: real 0.016s user 0.009s sys 0.008s
I20260810 11:35:37.278926 27512 ts_tablet_manager.cc:1428] T 2bd59bb23f284886ba9ef7883f361ab3 P 1797c22511034100a7414a5afd670ae6: Time spent starting tablet: real 0.021s user 0.019s sys 0.001s
W20260810 11:35:37.356482 27493 tablet.cc:2369] T 2bd59bb23f284886ba9ef7883f361ab3 P 312a2553119d4f37b3d724e39d3cdaff: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 11:35:37.408504 27219 tablet.cc:2369] T 2bd59bb23f284886ba9ef7883f361ab3 P 1797c22511034100a7414a5afd670ae6: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 11:35:37.509987 27356 tablet.cc:2369] T 2bd59bb23f284886ba9ef7883f361ab3 P 78a178dd3aa744e3abbf551a85de44bf: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 11:35:37.539849 27519 raft_consensus.cc:482] T 2bd59bb23f284886ba9ef7883f361ab3 P 1797c22511034100a7414a5afd670ae6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 11:35:37.540321 27519 raft_consensus.cc:504] T 2bd59bb23f284886ba9ef7883f361ab3 P 1797c22511034100a7414a5afd670ae6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "312a2553119d4f37b3d724e39d3cdaff" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 40593 } } peers { permanent_uuid: "78a178dd3aa744e3abbf551a85de44bf" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 42405 } } peers { permanent_uuid: "1797c22511034100a7414a5afd670ae6" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 36015 } }
I20260810 11:35:37.542555 27519 leader_election.cc:288] T 2bd59bb23f284886ba9ef7883f361ab3 P 1797c22511034100a7414a5afd670ae6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 312a2553119d4f37b3d724e39d3cdaff (127.8.18.195:40593), 78a178dd3aa744e3abbf551a85de44bf (127.8.18.194:42405)
I20260810 11:35:37.550565 27507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:37.543140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.193:53851 (local address 127.8.18.195:40593)
0810 11:35:37.543365 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0810 11:35:37.543372 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:35:37.544064 (+ 692us) server_negotiation.cc:407] Connection header received
0810 11:35:37.544355 (+ 291us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:35:37.544362 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:35:37.544501 (+ 139us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:35:37.544613 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:35:37.545389 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:37.546236 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:37.548420 (+ 2184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:37.549368 (+ 948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:37.549633 (+ 265us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:35:37.549715 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0810 11:35:37.550103 (+ 388us) server_negotiation.cc:299] Negotiation successful
0810 11:35:37.550277 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260810 11:35:37.550684 27521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:37.543000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.195:40593 (local address 127.8.18.193:53851)
0810 11:35:37.543928 (+ 928us) negotiation.cc:107] Waiting for socket to connect
0810 11:35:37.543947 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 11:35:37.544151 (+ 204us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:35:37.544635 (+ 484us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:35:37.544641 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:35:37.544675 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:35:37.545162 (+ 487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:35:37.545180 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:35:37.546440 (+ 1260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:35:37.546446 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:35:37.548208 (+ 1762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:35:37.548223 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:35:37.549651 (+ 1428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:35:37.549657 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:35:37.549862 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:35:37.549905 (+ 43us) client_negotiation.cc:769] Sending connection context
0810 11:35:37.550020 (+ 115us) client_negotiation.cc:240] Negotiation successful
0810 11:35:37.550210 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":769,"thread_start_us":514,"threads_started":1}
I20260810 11:35:37.550715 27522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:37.543199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.194:42405 (local address 127.8.18.193:44165)
0810 11:35:37.544198 (+ 999us) negotiation.cc:107] Waiting for socket to connect
0810 11:35:37.544224 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0810 11:35:37.544379 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:35:37.544932 (+ 553us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:35:37.544937 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:35:37.544967 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:35:37.545503 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:35:37.545520 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:35:37.546868 (+ 1348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:35:37.546873 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:35:37.548458 (+ 1585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:35:37.548475 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:35:37.549894 (+ 1419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:35:37.549900 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:35:37.550069 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:35:37.550095 (+ 26us) client_negotiation.cc:769] Sending connection context
0810 11:35:37.550181 (+ 86us) client_negotiation.cc:240] Negotiation successful
0810 11:35:37.550287 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":870,"thread_start_us":606,"threads_started":1}
I20260810 11:35:37.551229 27509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:37.543200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.193:44165 (local address 127.8.18.194:42405)
0810 11:35:37.543592 (+ 392us) server_negotiation.cc:206] Beginning negotiation
0810 11:35:37.543601 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 11:35:37.544314 (+ 713us) server_negotiation.cc:407] Connection header received
0810 11:35:37.544616 (+ 302us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:35:37.544660 (+ 44us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:35:37.544782 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:35:37.544936 (+ 154us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:35:37.545728 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:37.546689 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:37.548681 (+ 1992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:37.549698 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:37.549832 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:35:37.549886 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0810 11:35:37.550238 (+ 352us) server_negotiation.cc:299] Negotiation successful
0810 11:35:37.550377 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":217}
I20260810 11:35:37.551920 27444 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2bd59bb23f284886ba9ef7883f361ab3" candidate_uuid: "1797c22511034100a7414a5afd670ae6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "312a2553119d4f37b3d724e39d3cdaff" is_pre_election: true
I20260810 11:35:37.552354 27308 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2bd59bb23f284886ba9ef7883f361ab3" candidate_uuid: "1797c22511034100a7414a5afd670ae6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "78a178dd3aa744e3abbf551a85de44bf" is_pre_election: true
I20260810 11:35:37.552579 27444 raft_consensus.cc:2454] T 2bd59bb23f284886ba9ef7883f361ab3 P 312a2553119d4f37b3d724e39d3cdaff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1797c22511034100a7414a5afd670ae6 in term 0.
I20260810 11:35:37.552971 27308 raft_consensus.cc:2454] T 2bd59bb23f284886ba9ef7883f361ab3 P 78a178dd3aa744e3abbf551a85de44bf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1797c22511034100a7414a5afd670ae6 in term 0.
I20260810 11:35:37.553409 27100 leader_election.cc:302] T 2bd59bb23f284886ba9ef7883f361ab3 P 1797c22511034100a7414a5afd670ae6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1797c22511034100a7414a5afd670ae6, 312a2553119d4f37b3d724e39d3cdaff; no voters:
I20260810 11:35:37.554020 27519 raft_consensus.cc:2781] T 2bd59bb23f284886ba9ef7883f361ab3 P 1797c22511034100a7414a5afd670ae6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 11:35:37.554198 27519 raft_consensus.cc:482] T 2bd59bb23f284886ba9ef7883f361ab3 P 1797c22511034100a7414a5afd670ae6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 11:35:37.554330 27519 raft_consensus.cc:3037] T 2bd59bb23f284886ba9ef7883f361ab3 P 1797c22511034100a7414a5afd670ae6 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:35:37.556658 27519 raft_consensus.cc:504] T 2bd59bb23f284886ba9ef7883f361ab3 P 1797c22511034100a7414a5afd670ae6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "312a2553119d4f37b3d724e39d3cdaff" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 40593 } } peers { permanent_uuid: "78a178dd3aa744e3abbf551a85de44bf" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 42405 } } peers { permanent_uuid: "1797c22511034100a7414a5afd670ae6" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 36015 } }
I20260810 11:35:37.557436 27519 leader_election.cc:288] T 2bd59bb23f284886ba9ef7883f361ab3 P 1797c22511034100a7414a5afd670ae6 [CANDIDATE]: Term 1 election: Requested vote from peers 312a2553119d4f37b3d724e39d3cdaff (127.8.18.195:40593), 78a178dd3aa744e3abbf551a85de44bf (127.8.18.194:42405)
I20260810 11:35:37.558156 27308 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2bd59bb23f284886ba9ef7883f361ab3" candidate_uuid: "1797c22511034100a7414a5afd670ae6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "78a178dd3aa744e3abbf551a85de44bf"
I20260810 11:35:37.558151 27444 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2bd59bb23f284886ba9ef7883f361ab3" candidate_uuid: "1797c22511034100a7414a5afd670ae6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "312a2553119d4f37b3d724e39d3cdaff"
I20260810 11:35:37.558427 27444 raft_consensus.cc:3037] T 2bd59bb23f284886ba9ef7883f361ab3 P 312a2553119d4f37b3d724e39d3cdaff [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:35:37.558427 27308 raft_consensus.cc:3037] T 2bd59bb23f284886ba9ef7883f361ab3 P 78a178dd3aa744e3abbf551a85de44bf [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:35:37.562242 27308 raft_consensus.cc:2454] T 2bd59bb23f284886ba9ef7883f361ab3 P 78a178dd3aa744e3abbf551a85de44bf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1797c22511034100a7414a5afd670ae6 in term 1.
I20260810 11:35:37.562242 27444 raft_consensus.cc:2454] T 2bd59bb23f284886ba9ef7883f361ab3 P 312a2553119d4f37b3d724e39d3cdaff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1797c22511034100a7414a5afd670ae6 in term 1.
I20260810 11:35:37.562914 27101 leader_election.cc:302] T 2bd59bb23f284886ba9ef7883f361ab3 P 1797c22511034100a7414a5afd670ae6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 1797c22511034100a7414a5afd670ae6, 312a2553119d4f37b3d724e39d3cdaff, 78a178dd3aa744e3abbf551a85de44bf; no voters:
I20260810 11:35:37.563347 27519 raft_consensus.cc:2781] T 2bd59bb23f284886ba9ef7883f361ab3 P 1797c22511034100a7414a5afd670ae6 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:35:37.563894 27519 raft_consensus.cc:686] T 2bd59bb23f284886ba9ef7883f361ab3 P 1797c22511034100a7414a5afd670ae6 [term 1 LEADER]: Becoming Leader. State: Replica: 1797c22511034100a7414a5afd670ae6, State: Running, Role: LEADER
I20260810 11:35:37.564455 27519 consensus_queue.cc:237] T 2bd59bb23f284886ba9ef7883f361ab3 P 1797c22511034100a7414a5afd670ae6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "312a2553119d4f37b3d724e39d3cdaff" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 40593 } } peers { permanent_uuid: "78a178dd3aa744e3abbf551a85de44bf" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 42405 } } peers { permanent_uuid: "1797c22511034100a7414a5afd670ae6" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 36015 } }
I20260810 11:35:37.570027 27019 catalog_manager.cc:5495] T 2bd59bb23f284886ba9ef7883f361ab3 P 1797c22511034100a7414a5afd670ae6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1797c22511034100a7414a5afd670ae6 (127.8.18.193). New cstate: current_term: 1 leader_uuid: "1797c22511034100a7414a5afd670ae6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "312a2553119d4f37b3d724e39d3cdaff" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 40593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "78a178dd3aa744e3abbf551a85de44bf" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 42405 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1797c22511034100a7414a5afd670ae6" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 36015 } health_report { overall_health: HEALTHY } } }
I20260810 11:35:37.664649 27510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:37.658696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40760 (local address 127.8.18.193:36015)
0810 11:35:37.658941 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0810 11:35:37.658946 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:35:37.659039 (+ 93us) server_negotiation.cc:407] Connection header received
0810 11:35:37.659275 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:35:37.659279 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:35:37.659418 (+ 139us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:35:37.659611 (+ 193us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 11:35:37.660022 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:37.660803 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:37.662296 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:37.662825 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:37.662948 (+ 123us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:35:37.663462 (+ 514us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 11:35:37.663988 (+ 526us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 11:35:37.664221 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0810 11:35:37.664386 (+ 165us) server_negotiation.cc:299] Negotiation successful
0810 11:35:37.664440 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105}
I20260810 11:35:37.674441 27308 raft_consensus.cc:1261] T 2bd59bb23f284886ba9ef7883f361ab3 P 78a178dd3aa744e3abbf551a85de44bf [term 1 FOLLOWER]: Refusing update from remote peer 1797c22511034100a7414a5afd670ae6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 11:35:37.674445 27444 raft_consensus.cc:1261] T 2bd59bb23f284886ba9ef7883f361ab3 P 312a2553119d4f37b3d724e39d3cdaff [term 1 FOLLOWER]: Refusing update from remote peer 1797c22511034100a7414a5afd670ae6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 11:35:37.675648 27523 consensus_queue.cc:1035] T 2bd59bb23f284886ba9ef7883f361ab3 P 1797c22511034100a7414a5afd670ae6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "78a178dd3aa744e3abbf551a85de44bf" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 42405 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 11:35:37.676106 27519 consensus_queue.cc:1035] T 2bd59bb23f284886ba9ef7883f361ab3 P 1797c22511034100a7414a5afd670ae6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "312a2553119d4f37b3d724e39d3cdaff" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 40593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 11:35:37.704946 27018 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48638:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 11:35:37.705279 27018 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48638:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 11:35:37.709643 27018 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8207377d1f6a4c9d8f07bd3cbd83ca78: Sending DeleteTablet for 3 replicas of tablet 2bd59bb23f284886ba9ef7883f361ab3
I20260810 11:35:37.711191 27424 tablet_service.cc:1514] Processing DeleteTablet for tablet 2bd59bb23f284886ba9ef7883f361ab3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:35:37 UTC) from {username='kudu', principal='kudu/127.8.18.254@KRBTEST.COM'} at 127.0.0.1:32846
I20260810 11:35:37.711267 27288 tablet_service.cc:1514] Processing DeleteTablet for tablet 2bd59bb23f284886ba9ef7883f361ab3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:35:37 UTC) from {username='kudu', principal='kudu/127.8.18.254@KRBTEST.COM'} at 127.0.0.1:57760
I20260810 11:35:37.712092 27150 tablet_service.cc:1514] Processing DeleteTablet for tablet 2bd59bb23f284886ba9ef7883f361ab3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:35:37 UTC) from {username='kudu', principal='kudu/127.8.18.254@KRBTEST.COM'} at 127.0.0.1:40756
I20260810 11:35:37.712370 27537 tablet_replica.cc:330] T 2bd59bb23f284886ba9ef7883f361ab3 P 312a2553119d4f37b3d724e39d3cdaff: stopping tablet replica
I20260810 11:35:37.712880 27537 raft_consensus.cc:2229] T 2bd59bb23f284886ba9ef7883f361ab3 P 312a2553119d4f37b3d724e39d3cdaff [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 11:35:37.713213 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 27084
I20260810 11:35:37.713485 27537 raft_consensus.cc:2258] T 2bd59bb23f284886ba9ef7883f361ab3 P 312a2553119d4f37b3d724e39d3cdaff [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:35:37.714582 27536 tablet_replica.cc:330] T 2bd59bb23f284886ba9ef7883f361ab3 P 78a178dd3aa744e3abbf551a85de44bf: stopping tablet replica
I20260810 11:35:37.715097 27536 raft_consensus.cc:2229] T 2bd59bb23f284886ba9ef7883f361ab3 P 78a178dd3aa744e3abbf551a85de44bf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 11:35:37.715713 27536 raft_consensus.cc:2258] T 2bd59bb23f284886ba9ef7883f361ab3 P 78a178dd3aa744e3abbf551a85de44bf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:35:37.716331 27537 ts_tablet_manager.cc:1905] T 2bd59bb23f284886ba9ef7883f361ab3 P 312a2553119d4f37b3d724e39d3cdaff: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:35:37.724220 27537 ts_tablet_manager.cc:1918] T 2bd59bb23f284886ba9ef7883f361ab3 P 312a2553119d4f37b3d724e39d3cdaff: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 11:35:37.724478 27537 log.cc:1196] T 2bd59bb23f284886ba9ef7883f361ab3 P 312a2553119d4f37b3d724e39d3cdaff: Deleting WAL directory at /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786361401922362-8267-0/minicluster-data/ts-2/wal/wals/2bd59bb23f284886ba9ef7883f361ab3
I20260810 11:35:37.725095 27537 ts_tablet_manager.cc:1939] T 2bd59bb23f284886ba9ef7883f361ab3 P 312a2553119d4f37b3d724e39d3cdaff: Deleting consensus metadata
I20260810 11:35:37.727865 27536 ts_tablet_manager.cc:1905] T 2bd59bb23f284886ba9ef7883f361ab3 P 78a178dd3aa744e3abbf551a85de44bf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:35:37.732709 27538 tablet_replica.cc:330] T 2bd59bb23f284886ba9ef7883f361ab3 P 1797c22511034100a7414a5afd670ae6: stopping tablet replica
I20260810 11:35:37.734786 27004 catalog_manager.cc:4841] TS 312a2553119d4f37b3d724e39d3cdaff (127.8.18.195:40593): tablet 2bd59bb23f284886ba9ef7883f361ab3 (table test-table [id=59b89d15cce14636979cb091456adf49]) successfully deleted
I20260810 11:35:37.733281 27538 raft_consensus.cc:2229] T 2bd59bb23f284886ba9ef7883f361ab3 P 1797c22511034100a7414a5afd670ae6 [term 1 LEADER]: Raft consensus shutting down.
I20260810 11:35:37.735984 27536 ts_tablet_manager.cc:1918] T 2bd59bb23f284886ba9ef7883f361ab3 P 78a178dd3aa744e3abbf551a85de44bf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 11:35:37.736187 27536 log.cc:1196] T 2bd59bb23f284886ba9ef7883f361ab3 P 78a178dd3aa744e3abbf551a85de44bf: Deleting WAL directory at /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786361401922362-8267-0/minicluster-data/ts-1/wal/wals/2bd59bb23f284886ba9ef7883f361ab3
I20260810 11:35:37.736400 27538 raft_consensus.cc:2258] T 2bd59bb23f284886ba9ef7883f361ab3 P 1797c22511034100a7414a5afd670ae6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:35:37.736865 27536 ts_tablet_manager.cc:1939] T 2bd59bb23f284886ba9ef7883f361ab3 P 78a178dd3aa744e3abbf551a85de44bf: Deleting consensus metadata
I20260810 11:35:37.739051 27005 catalog_manager.cc:4841] TS 78a178dd3aa744e3abbf551a85de44bf (127.8.18.194:42405): tablet 2bd59bb23f284886ba9ef7883f361ab3 (table test-table [id=59b89d15cce14636979cb091456adf49]) successfully deleted
I20260810 11:35:37.739534 27538 ts_tablet_manager.cc:1905] T 2bd59bb23f284886ba9ef7883f361ab3 P 1797c22511034100a7414a5afd670ae6: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 10 11:35:37 dist-test-slave-srgv krb5kdc[26975](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361734, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.18.193@KRBTEST.COM
I20260810 11:35:37.749285 27538 ts_tablet_manager.cc:1918] T 2bd59bb23f284886ba9ef7883f361ab3 P 1797c22511034100a7414a5afd670ae6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 11:35:37.749572 27538 log.cc:1196] T 2bd59bb23f284886ba9ef7883f361ab3 P 1797c22511034100a7414a5afd670ae6: Deleting WAL directory at /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786361401922362-8267-0/minicluster-data/ts-0/wal/wals/2bd59bb23f284886ba9ef7883f361ab3
I20260810 11:35:37.750283 27538 ts_tablet_manager.cc:1939] T 2bd59bb23f284886ba9ef7883f361ab3 P 1797c22511034100a7414a5afd670ae6: Deleting consensus metadata
I20260810 11:35:37.752532 27005 catalog_manager.cc:4841] TS 1797c22511034100a7414a5afd670ae6 (127.8.18.193:36015): tablet 2bd59bb23f284886ba9ef7883f361ab3 (table test-table [id=59b89d15cce14636979cb091456adf49]) successfully deleted
I20260810 11:35:37.755355 27510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:37.717285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40770 (local address 127.8.18.193:36015)
0810 11:35:37.717584 (+ 299us) server_negotiation.cc:206] Beginning negotiation
0810 11:35:37.717590 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:35:37.728226 (+ 10636us) server_negotiation.cc:407] Connection header received
0810 11:35:37.728954 (+ 728us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:35:37.728964 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:35:37.729095 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:35:37.729226 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:35:37.733551 (+ 4325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:37.734950 (+ 1399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:37.737311 (+ 2361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:37.738273 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:37.738411 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:35:37.746082 (+ 7671us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:35:37.746103 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:35:37.746132 (+ 29us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:35:37.746220 (+ 88us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:35:37.751566 (+ 5346us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:35:37.752741 (+ 1175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:35:37.752752 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:35:37.752755 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:35:37.752998 (+ 243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:35:37.753555 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:35:37.753559 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:35:37.753562 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:35:37.753982 (+ 420us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:35:37.754241 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0810 11:35:37.754710 (+ 469us) server_negotiation.cc:299] Negotiation successful
0810 11:35:37.754936 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260810 11:35:37.756162 27212 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:35:37.863693 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 27084
I20260810 11:35:37.885458 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 27224
Aug 10 11:35:37 dist-test-slave-srgv krb5kdc[26975](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361734, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.18.194@KRBTEST.COM
I20260810 11:35:37.903604 27509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:37.886363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57768 (local address 127.8.18.194:42405)
0810 11:35:37.886603 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0810 11:35:37.886611 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:35:37.886635 (+ 24us) server_negotiation.cc:407] Connection header received
0810 11:35:37.886855 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:35:37.886868 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:35:37.886961 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:35:37.887116 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:35:37.888532 (+ 1416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:37.889241 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:37.890824 (+ 1583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:37.891280 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:37.891436 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:35:37.896569 (+ 5133us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:35:37.896591 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:35:37.896609 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:35:37.896655 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:35:37.900084 (+ 3429us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:35:37.901118 (+ 1034us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:35:37.901128 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:35:37.901131 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:35:37.901288 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:35:37.901772 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:35:37.901780 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:35:37.901788 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:35:37.902222 (+ 434us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:35:37.902569 (+ 347us) server_negotiation.cc:1035] Waiting for connection context
0810 11:35:37.902952 (+ 383us) server_negotiation.cc:299] Negotiation successful
0810 11:35:37.903226 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260810 11:35:37.904417 27350 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:35:38.003219 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 27224
I20260810 11:35:38.024648 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 27361
Aug 10 11:35:38 dist-test-slave-srgv krb5kdc[26975](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361734, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.18.195@KRBTEST.COM
I20260810 11:35:38.042789 27507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:38.025587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32850 (local address 127.8.18.195:40593)
0810 11:35:38.025794 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0810 11:35:38.025798 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 11:35:38.025820 (+ 22us) server_negotiation.cc:407] Connection header received
0810 11:35:38.025926 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:35:38.025931 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:35:38.026001 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:35:38.026095 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:35:38.027626 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:38.028469 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:38.030005 (+ 1536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:38.030395 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:38.030500 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:35:38.036275 (+ 5775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:35:38.036295 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:35:38.036312 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:35:38.036364 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:35:38.039454 (+ 3090us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:35:38.040380 (+ 926us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:35:38.040389 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:35:38.040391 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:35:38.040557 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:35:38.040985 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:35:38.040990 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:35:38.040992 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:35:38.041568 (+ 576us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:35:38.041910 (+ 342us) server_negotiation.cc:1035] Waiting for connection context
0810 11:35:38.042203 (+ 293us) server_negotiation.cc:299] Negotiation successful
0810 11:35:38.042386 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260810 11:35:38.043466 27487 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:35:38.137787 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 27361
I20260810 11:35:38.159291 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 26988
I20260810 11:35:38.160578 27053 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:35:38.293030 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 26988
2026-08-10T11:35:38Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (5075 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-10T11:35:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T11:35:38Z Disabled control of system clock
I20260810 11:35:38.389979 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786361401922362-8267-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786361401922362-8267-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786361401922362-8267-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.18.254:35041
--webserver_interface=127.8.18.254
--webserver_port=0
--builtin_ntp_servers=127.8.18.212:43881
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.18.254:35041
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 11:35:38.773998 27547 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:35:38.774340 27547 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:35:38.774430 27547 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:35:38.782559 27547 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 11:35:38.782694 27547 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:35:38.782734 27547 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:35:38.782796 27547 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 11:35:38.782841 27547 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 11:35:38.793249 27547 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.18.212:43881
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786361401922362-8267-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786361401922362-8267-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.18.254:35041
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.18.254:35041
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.18.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786361401922362-8267-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:35:38.794684 27547 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:35:38.796645 27547 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:35:38.810678 27553 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:35:38.810528 27555 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:35:38.810660 27552 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:35:38.811483 27547 server_base.cc:1034] running on GCE node
I20260810 11:35:38.812496 27547 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:35:38.814354 27547 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:35:38.815670 27547 hybrid_clock.cc:648] HybridClock initialized: now 1786361738815605 us; error 69 us; skew 500 ppm
I20260810 11:35:38.816100 27547 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 11:35:38.819036 27547 webserver.cc:460] Webserver started at http://127.8.18.254:39571/ using document root <none> and password file <none>
I20260810 11:35:38.819890 27547 fs_manager.cc:359] Metadata directory not provided
I20260810 11:35:38.820047 27547 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:35:38.820410 27547 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:35:38.823815 27547 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786361401922362-8267-0/minicluster-data/master-0/data/instance:
uuid: "d46b9d4a638641f3a59dac2a63c23b45"
format_stamp: "Formatted at 2026-08-10 11:35:38 on dist-test-slave-srgv"
server_key: "8d0a68669e3a7b6a5570ec75fe88c271"
server_key_iv: "958d3cc93c22f97e3f0c214805288369"
server_key_version: "encryptionkey@0"
I20260810 11:35:38.824820 27547 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786361401922362-8267-0/minicluster-data/master-0/wal/instance:
uuid: "d46b9d4a638641f3a59dac2a63c23b45"
format_stamp: "Formatted at 2026-08-10 11:35:38 on dist-test-slave-srgv"
server_key: "8d0a68669e3a7b6a5570ec75fe88c271"
server_key_iv: "958d3cc93c22f97e3f0c214805288369"
server_key_version: "encryptionkey@0"
I20260810 11:35:38.830668 27547 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260810 11:35:38.834599 27561 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:35:38.836256 27547 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260810 11:35:38.836472 27547 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786361401922362-8267-0/minicluster-data/master-0/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786361401922362-8267-0/minicluster-data/master-0/wal
uuid: "d46b9d4a638641f3a59dac2a63c23b45"
format_stamp: "Formatted at 2026-08-10 11:35:38 on dist-test-slave-srgv"
server_key: "8d0a68669e3a7b6a5570ec75fe88c271"
server_key_iv: "958d3cc93c22f97e3f0c214805288369"
server_key_version: "encryptionkey@0"
I20260810 11:35:38.836756 27547 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786361401922362-8267-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786361401922362-8267-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786361401922362-8267-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:35:38.855690 27547 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:35:38.856642 27547 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:35:38.856971 27547 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:35:38.878963 27547 rpc_server.cc:307] RPC server started. Bound to: 127.8.18.254:35041
I20260810 11:35:38.878981 27614 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.18.254:35041 every 8 connection(s)
I20260810 11:35:38.880532 27547 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
I20260810 11:35:38.884572 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 27547
I20260810 11:35:38.884761 8267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786361401922362-8267-0/minicluster-data/master-0/wal/instance
I20260810 11:35:38.885272 27615 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:35:38.885326 8267 external_mini_cluster.cc:1428] Setting key a720424cb41051407f5ac65fd4a2e85b
I20260810 11:35:38.901008 27615 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d46b9d4a638641f3a59dac2a63c23b45: Bootstrap starting.
I20260810 11:35:38.901697 27618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:38.887497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43716 (local address 127.8.18.254:35041)
0810 11:35:38.888703 (+ 1206us) server_negotiation.cc:206] Beginning negotiation
0810 11:35:38.888714 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 11:35:38.888767 (+ 53us) server_negotiation.cc:407] Connection header received
0810 11:35:38.890561 (+ 1794us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:35:38.890600 (+ 39us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:35:38.891152 (+ 552us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:35:38.891533 (+ 381us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:35:38.892321 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:38.893875 (+ 1554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:38.895972 (+ 2097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:38.896840 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:38.897004 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:35:38.898528 (+ 1524us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:35:38.898541 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:35:38.898559 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:35:38.899260 (+ 701us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:35:38.899350 (+ 90us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:35:38.899375 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:35:38.899485 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0810 11:35:38.899796 (+ 311us) server_negotiation.cc:299] Negotiation successful
0810 11:35:38.900265 (+ 469us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":661,"thread_start_us":429,"threads_started":1}
I20260810 11:35:38.905200 27615 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d46b9d4a638641f3a59dac2a63c23b45: Neither blocks nor log segments found. Creating new log.
I20260810 11:35:38.906428 27615 log.cc:824] T 00000000000000000000000000000000 P d46b9d4a638641f3a59dac2a63c23b45: Log is configured to *not* fsync() on all Append() calls
I20260810 11:35:38.909945 27615 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d46b9d4a638641f3a59dac2a63c23b45: No bootstrap required, opened a new log
I20260810 11:35:38.918273 27615 raft_consensus.cc:348] T 00000000000000000000000000000000 P d46b9d4a638641f3a59dac2a63c23b45 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d46b9d4a638641f3a59dac2a63c23b45" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 35041 } }
I20260810 11:35:38.918852 27615 raft_consensus.cc:374] T 00000000000000000000000000000000 P d46b9d4a638641f3a59dac2a63c23b45 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:35:38.918982 27615 raft_consensus.cc:729] T 00000000000000000000000000000000 P d46b9d4a638641f3a59dac2a63c23b45 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d46b9d4a638641f3a59dac2a63c23b45, State: Initialized, Role: FOLLOWER
I20260810 11:35:38.919687 27615 consensus_queue.cc:260] T 00000000000000000000000000000000 P d46b9d4a638641f3a59dac2a63c23b45 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d46b9d4a638641f3a59dac2a63c23b45" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 35041 } }
I20260810 11:35:38.919956 27615 raft_consensus.cc:388] T 00000000000000000000000000000000 P d46b9d4a638641f3a59dac2a63c23b45 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 11:35:38.920112 27615 raft_consensus.cc:482] T 00000000000000000000000000000000 P d46b9d4a638641f3a59dac2a63c23b45 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 11:35:38.920305 27615 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d46b9d4a638641f3a59dac2a63c23b45 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:35:38.922811 27615 raft_consensus.cc:504] T 00000000000000000000000000000000 P d46b9d4a638641f3a59dac2a63c23b45 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d46b9d4a638641f3a59dac2a63c23b45" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 35041 } }
I20260810 11:35:38.923448 27615 leader_election.cc:302] T 00000000000000000000000000000000 P d46b9d4a638641f3a59dac2a63c23b45 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d46b9d4a638641f3a59dac2a63c23b45; no voters:
I20260810 11:35:38.924243 27615 leader_election.cc:288] T 00000000000000000000000000000000 P d46b9d4a638641f3a59dac2a63c23b45 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 11:35:38.924376 27620 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d46b9d4a638641f3a59dac2a63c23b45 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:35:38.925200 27620 raft_consensus.cc:686] T 00000000000000000000000000000000 P d46b9d4a638641f3a59dac2a63c23b45 [term 1 LEADER]: Becoming Leader. State: Replica: d46b9d4a638641f3a59dac2a63c23b45, State: Running, Role: LEADER
I20260810 11:35:38.925834 27620 consensus_queue.cc:237] T 00000000000000000000000000000000 P d46b9d4a638641f3a59dac2a63c23b45 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d46b9d4a638641f3a59dac2a63c23b45" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 35041 } }
I20260810 11:35:38.926839 27615 sys_catalog.cc:564] T 00000000000000000000000000000000 P d46b9d4a638641f3a59dac2a63c23b45 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 11:35:38.929141 27621 sys_catalog.cc:455] T 00000000000000000000000000000000 P d46b9d4a638641f3a59dac2a63c23b45 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d46b9d4a638641f3a59dac2a63c23b45" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d46b9d4a638641f3a59dac2a63c23b45" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 35041 } } }
I20260810 11:35:38.929679 27621 sys_catalog.cc:458] T 00000000000000000000000000000000 P d46b9d4a638641f3a59dac2a63c23b45 [sys.catalog]: This master's current role is: LEADER
I20260810 11:35:38.930521 27622 sys_catalog.cc:455] T 00000000000000000000000000000000 P d46b9d4a638641f3a59dac2a63c23b45 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d46b9d4a638641f3a59dac2a63c23b45. Latest consensus state: current_term: 1 leader_uuid: "d46b9d4a638641f3a59dac2a63c23b45" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d46b9d4a638641f3a59dac2a63c23b45" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 35041 } } }
I20260810 11:35:38.930878 27622 sys_catalog.cc:458] T 00000000000000000000000000000000 P d46b9d4a638641f3a59dac2a63c23b45 [sys.catalog]: This master's current role is: LEADER
I20260810 11:35:38.932423 27627 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 11:35:38.939316 27627 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 11:35:38.949687 27627 catalog_manager.cc:1347] Generated new cluster ID: 8d34a94d46bb497b98671960c102b668
I20260810 11:35:38.949838 27627 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 11:35:38.971976 27627 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 11:35:38.973508 27627 catalog_manager.cc:1504] Loading token signing keys...
I20260810 11:35:38.992797 27627 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d46b9d4a638641f3a59dac2a63c23b45: Generated new TSK 0
I20260810 11:35:38.994115 27627 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 11:35:39.004808 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786361401922362-8267-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786361401922362-8267-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786361401922362-8267-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786361401922362-8267-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.18.193:0
--local_ip_for_outbound_sockets=127.8.18.193
--webserver_interface=127.8.18.193
--webserver_port=0
--tserver_master_addrs=127.8.18.254:35041
--builtin_ntp_servers=127.8.18.212:43881
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 11:35:39.382743 27639 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:35:39.383087 27639 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:35:39.383183 27639 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:35:39.391176 27639 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:35:39.391304 27639 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:35:39.391418 27639 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.18.193
I20260810 11:35:39.401098 27639 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.18.212:43881
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786361401922362-8267-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786361401922362-8267-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.18.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786361401922362-8267-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.18.193
--webserver_port=0
--tserver_master_addrs=127.8.18.254:35041
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.18.193
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786361401922362-8267-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:35:39.402640 27639 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:35:39.404520 27639 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:35:39.419826 27644 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:35:39.419881 27645 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:35:39.419827 27647 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:35:39.420888 27639 server_base.cc:1034] running on GCE node
I20260810 11:35:39.421839 27639 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:35:39.423470 27639 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:35:39.424767 27639 hybrid_clock.cc:648] HybridClock initialized: now 1786361739424702 us; error 55 us; skew 500 ppm
I20260810 11:35:39.425164 27639 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 11:35:39.427852 27639 webserver.cc:460] Webserver started at http://127.8.18.193:38465/ using document root <none> and password file <none>
I20260810 11:35:39.428710 27639 fs_manager.cc:359] Metadata directory not provided
I20260810 11:35:39.428859 27639 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:35:39.429242 27639 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:35:39.432688 27639 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786361401922362-8267-0/minicluster-data/ts-0/data/instance:
uuid: "7f5a9229ded94ae98545144a2859f2e5"
format_stamp: "Formatted at 2026-08-10 11:35:39 on dist-test-slave-srgv"
server_key: "815e9658a60eaa68d0a0210c2e60ad6a"
server_key_iv: "c32c09a0e21189d41f7a4ba08aaecd3f"
server_key_version: "encryptionkey@0"
I20260810 11:35:39.433772 27639 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786361401922362-8267-0/minicluster-data/ts-0/wal/instance:
uuid: "7f5a9229ded94ae98545144a2859f2e5"
format_stamp: "Formatted at 2026-08-10 11:35:39 on dist-test-slave-srgv"
server_key: "815e9658a60eaa68d0a0210c2e60ad6a"
server_key_iv: "c32c09a0e21189d41f7a4ba08aaecd3f"
server_key_version: "encryptionkey@0"
I20260810 11:35:39.440132 27639 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.001s sys 0.004s
I20260810 11:35:39.444860 27654 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:35:39.446650 27639 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260810 11:35:39.446908 27639 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786361401922362-8267-0/minicluster-data/ts-0/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786361401922362-8267-0/minicluster-data/ts-0/wal
uuid: "7f5a9229ded94ae98545144a2859f2e5"
format_stamp: "Formatted at 2026-08-10 11:35:39 on dist-test-slave-srgv"
server_key: "815e9658a60eaa68d0a0210c2e60ad6a"
server_key_iv: "c32c09a0e21189d41f7a4ba08aaecd3f"
server_key_version: "encryptionkey@0"
I20260810 11:35:39.447237 27639 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786361401922362-8267-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786361401922362-8267-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786361401922362-8267-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:35:39.474795 27639 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:35:39.475883 27639 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:35:39.476302 27639 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:35:39.477651 27639 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:35:39.479410 27639 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:35:39.479550 27639 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:35:39.479697 27639 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:35:39.479802 27639 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:35:39.518939 27639 rpc_server.cc:307] RPC server started. Bound to: 127.8.18.193:43757
I20260810 11:35:39.519044 27771 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.18.193:43757 every 8 connection(s)
I20260810 11:35:39.520421 27639 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786361401922362-8267-0/minicluster-data/ts-0/data/info.pb
I20260810 11:35:39.529239 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 27639
I20260810 11:35:39.529425 8267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786361401922362-8267-0/minicluster-data/ts-0/wal/instance
I20260810 11:35:39.529953 8267 external_mini_cluster.cc:1428] Setting key ab74bc728c248042fa8a0b26044a8740
I20260810 11:35:39.533314 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786361401922362-8267-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786361401922362-8267-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786361401922362-8267-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786361401922362-8267-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.18.194:0
--local_ip_for_outbound_sockets=127.8.18.194
--webserver_interface=127.8.18.194
--webserver_port=0
--tserver_master_addrs=127.8.18.254:35041
--builtin_ntp_servers=127.8.18.212:43881
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260810 11:35:39.536000 27774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:39.523394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.193:38941 (local address 127.8.18.254:35041)
0810 11:35:39.523939 (+ 545us) server_negotiation.cc:206] Beginning negotiation
0810 11:35:39.523946 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:35:39.525315 (+ 1369us) server_negotiation.cc:407] Connection header received
0810 11:35:39.527653 (+ 2338us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:35:39.527658 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:35:39.527742 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:35:39.527858 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:35:39.529287 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:39.530318 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:39.532607 (+ 2289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:39.533079 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:39.533239 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:35:39.535023 (+ 1784us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:35:39.535028 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:35:39.535032 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:35:39.535060 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:35:39.535105 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:35:39.535118 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:35:39.535230 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0810 11:35:39.535485 (+ 255us) server_negotiation.cc:299] Negotiation successful
0810 11:35:39.535604 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":407,"thread_start_us":249,"threads_started":1}
I20260810 11:35:39.537209 27775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:39.524062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:35041 (local address 127.8.18.193:38941)
0810 11:35:39.525111 (+ 1049us) negotiation.cc:107] Waiting for socket to connect
0810 11:35:39.525160 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0810 11:35:39.527235 (+ 2075us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:35:39.528241 (+ 1006us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:35:39.528252 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:35:39.528370 (+ 118us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:35:39.529042 (+ 672us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:35:39.529061 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:35:39.530526 (+ 1465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:35:39.530530 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:35:39.532433 (+ 1903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:35:39.532450 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:35:39.533379 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:35:39.533387 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:35:39.533773 (+ 386us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:35:39.534624 (+ 851us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 11:35:39.534627 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:35:39.534676 (+ 49us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 11:35:39.534776 (+ 100us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:35:39.535232 (+ 456us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:35:39.535248 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:35:39.535258 (+ 10us) client_negotiation.cc:769] Sending connection context
0810 11:35:39.535472 (+ 214us) client_negotiation.cc:240] Negotiation successful
0810 11:35:39.535952 (+ 480us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":546,"thread_start_us":328,"threads_started":1}
I20260810 11:35:39.538985 27772 heartbeater.cc:347] Connected to a master server at 127.8.18.254:35041
I20260810 11:35:39.539350 27772 heartbeater.cc:464] Registering TS with master...
I20260810 11:35:39.540395 27772 heartbeater.cc:511] Master 127.8.18.254:35041 requested a full tablet report, sending...
I20260810 11:35:39.542990 27578 ts_manager.cc:194] Registered new tserver with Master: 7f5a9229ded94ae98545144a2859f2e5 (127.8.18.193:43757)
I20260810 11:35:39.545161 27578 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.18.193:38941
W20260810 11:35:39.904052 27776 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:35:39.904450 27776 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:35:39.904632 27776 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:35:39.912483 27776 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:35:39.912712 27776 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:35:39.912910 27776 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.18.194
I20260810 11:35:39.922791 27776 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.18.212:43881
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786361401922362-8267-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786361401922362-8267-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.18.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786361401922362-8267-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.18.194
--webserver_port=0
--tserver_master_addrs=127.8.18.254:35041
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.18.194
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786361401922362-8267-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:35:39.924242 27776 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:35:39.926159 27776 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:35:39.941021 27781 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:35:39.941018 27782 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:35:39.940953 27784 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:35:39.942909 27776 server_base.cc:1034] running on GCE node
I20260810 11:35:39.943763 27776 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:35:39.945333 27776 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:35:39.946662 27776 hybrid_clock.cc:648] HybridClock initialized: now 1786361739946568 us; error 68 us; skew 500 ppm
I20260810 11:35:39.947134 27776 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 11:35:39.950599 27776 webserver.cc:460] Webserver started at http://127.8.18.194:46773/ using document root <none> and password file <none>
I20260810 11:35:39.951669 27776 fs_manager.cc:359] Metadata directory not provided
I20260810 11:35:39.951846 27776 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:35:39.952306 27776 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:35:39.956064 27776 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786361401922362-8267-0/minicluster-data/ts-1/data/instance:
uuid: "918a4750cfb44a89ac206d5a43bb91d1"
format_stamp: "Formatted at 2026-08-10 11:35:39 on dist-test-slave-srgv"
server_key: "cbcae75fea115f8db3a76987d1b1ccf7"
server_key_iv: "c41aeeb1488c91abf14baa751cb6c016"
server_key_version: "encryptionkey@0"
I20260810 11:35:39.957022 27776 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786361401922362-8267-0/minicluster-data/ts-1/wal/instance:
uuid: "918a4750cfb44a89ac206d5a43bb91d1"
format_stamp: "Formatted at 2026-08-10 11:35:39 on dist-test-slave-srgv"
server_key: "cbcae75fea115f8db3a76987d1b1ccf7"
server_key_iv: "c41aeeb1488c91abf14baa751cb6c016"
server_key_version: "encryptionkey@0"
I20260810 11:35:39.962929 27776 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260810 11:35:39.967231 27790 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:35:39.968820 27776 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 11:35:39.969024 27776 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786361401922362-8267-0/minicluster-data/ts-1/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786361401922362-8267-0/minicluster-data/ts-1/wal
uuid: "918a4750cfb44a89ac206d5a43bb91d1"
format_stamp: "Formatted at 2026-08-10 11:35:39 on dist-test-slave-srgv"
server_key: "cbcae75fea115f8db3a76987d1b1ccf7"
server_key_iv: "c41aeeb1488c91abf14baa751cb6c016"
server_key_version: "encryptionkey@0"
I20260810 11:35:39.969300 27776 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786361401922362-8267-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786361401922362-8267-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786361401922362-8267-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:35:39.996960 27776 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:35:39.998068 27776 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:35:39.998417 27776 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:35:39.999519 27776 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:35:40.001233 27776 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:35:40.001333 27776 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:35:40.001524 27776 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:35:40.001616 27776 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:35:40.037802 27776 rpc_server.cc:307] RPC server started. Bound to: 127.8.18.194:36477
I20260810 11:35:40.037917 27905 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.18.194:36477 every 8 connection(s)
I20260810 11:35:40.039287 27776 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786361401922362-8267-0/minicluster-data/ts-1/data/info.pb
I20260810 11:35:40.049238 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 27776
I20260810 11:35:40.049391 8267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786361401922362-8267-0/minicluster-data/ts-1/wal/instance
I20260810 11:35:40.049930 8267 external_mini_cluster.cc:1428] Setting key e1e0cd75c03b75a7998d43adfb9be6dd
I20260810 11:35:40.053382 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786361401922362-8267-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786361401922362-8267-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786361401922362-8267-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786361401922362-8267-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.18.195:0
--local_ip_for_outbound_sockets=127.8.18.195
--webserver_interface=127.8.18.195
--webserver_port=0
--tserver_master_addrs=127.8.18.254:35041
--builtin_ntp_servers=127.8.18.212:43881
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260810 11:35:40.054378 27908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:40.042322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.194:45039 (local address 127.8.18.254:35041)
0810 11:35:40.042834 (+ 512us) server_negotiation.cc:206] Beginning negotiation
0810 11:35:40.042839 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:35:40.044201 (+ 1362us) server_negotiation.cc:407] Connection header received
0810 11:35:40.046382 (+ 2181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:35:40.046388 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:35:40.046490 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:35:40.046629 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:35:40.047914 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:40.048674 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:40.050637 (+ 1963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:40.051295 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:40.051449 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:35:40.053369 (+ 1920us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:35:40.053375 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:35:40.053378 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:35:40.053398 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:35:40.053502 (+ 104us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:35:40.053516 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:35:40.053626 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0810 11:35:40.053878 (+ 252us) server_negotiation.cc:299] Negotiation successful
0810 11:35:40.053993 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":401,"thread_start_us":276,"threads_started":1}
I20260810 11:35:40.055505 27909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:40.042858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:35041 (local address 127.8.18.194:45039)
0810 11:35:40.043918 (+ 1060us) negotiation.cc:107] Waiting for socket to connect
0810 11:35:40.043958 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0810 11:35:40.046024 (+ 2066us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:35:40.046918 (+ 894us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:35:40.046928 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:35:40.047041 (+ 113us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:35:40.047700 (+ 659us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:35:40.047720 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:35:40.048861 (+ 1141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:35:40.048865 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:35:40.050454 (+ 1589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:35:40.050477 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:35:40.051586 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:35:40.051596 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:35:40.052009 (+ 413us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:35:40.052946 (+ 937us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 11:35:40.052950 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:35:40.053006 (+ 56us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 11:35:40.053112 (+ 106us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:35:40.053627 (+ 515us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:35:40.053643 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:35:40.053654 (+ 11us) client_negotiation.cc:769] Sending connection context
0810 11:35:40.053862 (+ 208us) client_negotiation.cc:240] Negotiation successful
0810 11:35:40.054331 (+ 469us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":571,"thread_start_us":348,"threads_started":1}
I20260810 11:35:40.057156 27906 heartbeater.cc:347] Connected to a master server at 127.8.18.254:35041
I20260810 11:35:40.057555 27906 heartbeater.cc:464] Registering TS with master...
I20260810 11:35:40.058413 27906 heartbeater.cc:511] Master 127.8.18.254:35041 requested a full tablet report, sending...
I20260810 11:35:40.060149 27578 ts_manager.cc:194] Registered new tserver with Master: 918a4750cfb44a89ac206d5a43bb91d1 (127.8.18.194:36477)
I20260810 11:35:40.061448 27578 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.18.194:45039
W20260810 11:35:40.425534 27910 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:35:40.426028 27910 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:35:40.426223 27910 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:35:40.435149 27910 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:35:40.435330 27910 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:35:40.435527 27910 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.18.195
I20260810 11:35:40.447079 27910 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.18.212:43881
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786361401922362-8267-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786361401922362-8267-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.18.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786361401922362-8267-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.18.195
--webserver_port=0
--tserver_master_addrs=127.8.18.254:35041
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.18.195
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786361401922362-8267-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:35:40.448587 27910 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:35:40.450618 27910 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:35:40.465067 27916 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:35:40.465107 27917 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:35:40.465397 27919 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:35:40.465950 27910 server_base.cc:1034] running on GCE node
I20260810 11:35:40.466720 27910 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:35:40.467825 27910 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:35:40.469082 27910 hybrid_clock.cc:648] HybridClock initialized: now 1786361740469045 us; error 49 us; skew 500 ppm
I20260810 11:35:40.469542 27910 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 11:35:40.472198 27910 webserver.cc:460] Webserver started at http://127.8.18.195:38707/ using document root <none> and password file <none>
I20260810 11:35:40.473067 27910 fs_manager.cc:359] Metadata directory not provided
I20260810 11:35:40.473233 27910 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:35:40.473776 27910 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:35:40.477200 27910 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786361401922362-8267-0/minicluster-data/ts-2/data/instance:
uuid: "df43fc9333754a9db8905fd393d4aa31"
format_stamp: "Formatted at 2026-08-10 11:35:40 on dist-test-slave-srgv"
server_key: "9967d33bae90f62b368ea344507afb19"
server_key_iv: "5bf2cca4b2eb2f4a061045d8c8a29526"
server_key_version: "encryptionkey@0"
I20260810 11:35:40.478247 27910 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786361401922362-8267-0/minicluster-data/ts-2/wal/instance:
uuid: "df43fc9333754a9db8905fd393d4aa31"
format_stamp: "Formatted at 2026-08-10 11:35:40 on dist-test-slave-srgv"
server_key: "9967d33bae90f62b368ea344507afb19"
server_key_iv: "5bf2cca4b2eb2f4a061045d8c8a29526"
server_key_version: "encryptionkey@0"
I20260810 11:35:40.483922 27910 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260810 11:35:40.487975 27925 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:35:40.489610 27910 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260810 11:35:40.489800 27910 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786361401922362-8267-0/minicluster-data/ts-2/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786361401922362-8267-0/minicluster-data/ts-2/wal
uuid: "df43fc9333754a9db8905fd393d4aa31"
format_stamp: "Formatted at 2026-08-10 11:35:40 on dist-test-slave-srgv"
server_key: "9967d33bae90f62b368ea344507afb19"
server_key_iv: "5bf2cca4b2eb2f4a061045d8c8a29526"
server_key_version: "encryptionkey@0"
I20260810 11:35:40.490068 27910 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786361401922362-8267-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786361401922362-8267-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786361401922362-8267-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:35:40.504364 27910 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:35:40.505313 27910 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:35:40.505679 27910 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:35:40.506788 27910 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:35:40.508525 27910 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:35:40.508633 27910 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:35:40.508832 27910 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:35:40.508909 27910 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:35:40.548686 27910 rpc_server.cc:307] RPC server started. Bound to: 127.8.18.195:39803
I20260810 11:35:40.548751 28041 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.18.195:39803 every 8 connection(s)
I20260810 11:35:40.549679 27772 heartbeater.cc:503] Master 127.8.18.254:35041 was elected leader, sending a full tablet report...
I20260810 11:35:40.550635 27910 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786361401922362-8267-0/minicluster-data/ts-2/data/info.pb
I20260810 11:35:40.558596 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 27910
I20260810 11:35:40.558802 8267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786361401922362-8267-0/minicluster-data/ts-2/wal/instance
I20260810 11:35:40.559288 8267 external_mini_cluster.cc:1428] Setting key b34df91184badc011ca4896e7a50d133
I20260810 11:35:40.565176 27908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:40.553672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.195:57879 (local address 127.8.18.254:35041)
0810 11:35:40.553923 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0810 11:35:40.553931 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:35:40.555446 (+ 1515us) server_negotiation.cc:407] Connection header received
0810 11:35:40.557279 (+ 1833us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:35:40.557284 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:35:40.557355 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:35:40.557530 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:35:40.558923 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:40.559845 (+ 922us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:40.562156 (+ 2311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:40.562639 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:40.562753 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:35:40.564430 (+ 1677us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:35:40.564435 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:35:40.564438 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:35:40.564458 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:35:40.564493 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:35:40.564505 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:35:40.564582 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0810 11:35:40.564831 (+ 249us) server_negotiation.cc:299] Negotiation successful
0810 11:35:40.564920 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102}
I20260810 11:35:40.566293 28044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:40.554293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:35041 (local address 127.8.18.195:57879)
0810 11:35:40.555272 (+ 979us) negotiation.cc:107] Waiting for socket to connect
0810 11:35:40.555315 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 11:35:40.556968 (+ 1653us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:35:40.557734 (+ 766us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:35:40.557743 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:35:40.557832 (+ 89us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:35:40.558688 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:35:40.558716 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:35:40.560016 (+ 1300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:35:40.560019 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:35:40.561965 (+ 1946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:35:40.561980 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:35:40.562883 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:35:40.562889 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:35:40.563200 (+ 311us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:35:40.564103 (+ 903us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 11:35:40.564106 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:35:40.564151 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 11:35:40.564238 (+ 87us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:35:40.564625 (+ 387us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:35:40.564634 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:35:40.564641 (+ 7us) client_negotiation.cc:769] Sending connection context
0810 11:35:40.564798 (+ 157us) client_negotiation.cc:240] Negotiation successful
0810 11:35:40.565126 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":526,"thread_start_us":336,"threads_started":1}
I20260810 11:35:40.567873 28042 heartbeater.cc:347] Connected to a master server at 127.8.18.254:35041
I20260810 11:35:40.568337 28042 heartbeater.cc:464] Registering TS with master...
I20260810 11:35:40.569253 28042 heartbeater.cc:511] Master 127.8.18.254:35041 requested a full tablet report, sending...
I20260810 11:35:40.571012 27578 ts_manager.cc:194] Registered new tserver with Master: df43fc9333754a9db8905fd393d4aa31 (127.8.18.195:39803)
I20260810 11:35:40.571817 27578 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.18.195:57879
I20260810 11:35:40.575299 8267 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 11:35:40.586352 27908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:40.579925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43720 (local address 127.8.18.254:35041)
0810 11:35:40.580126 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0810 11:35:40.580130 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 11:35:40.580654 (+ 524us) server_negotiation.cc:407] Connection header received
0810 11:35:40.580799 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:35:40.580803 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:35:40.580889 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:35:40.580996 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:35:40.581665 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:40.582417 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:40.583874 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:40.584343 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:40.584450 (+ 107us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:35:40.585647 (+ 1197us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:35:40.585652 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:35:40.585655 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:35:40.585676 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:35:40.585708 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:35:40.585720 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:35:40.585846 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0810 11:35:40.586021 (+ 175us) server_negotiation.cc:299] Negotiation successful
0810 11:35:40.586072 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260810 11:35:40.590634 27578 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43720:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 11:35:40.598073 27578 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 11:35:40.623986 28057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:40.612375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48312 (local address 127.8.18.193:43757)
0810 11:35:40.615861 (+ 3486us) server_negotiation.cc:206] Beginning negotiation
0810 11:35:40.615869 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:35:40.615900 (+ 31us) server_negotiation.cc:407] Connection header received
0810 11:35:40.615999 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:35:40.616008 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:35:40.616268 (+ 260us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:35:40.616456 (+ 188us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:35:40.617595 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:40.618660 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:40.621277 (+ 2617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:40.622693 (+ 1416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:40.622835 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:35:40.623174 (+ 339us) server_negotiation.cc:1035] Waiting for connection context
0810 11:35:40.623388 (+ 214us) server_negotiation.cc:299] Negotiation successful
0810 11:35:40.623574 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3303,"thread_start_us":2550,"threads_started":1}
I20260810 11:35:40.625074 28055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:40.612987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.195:39803 (local address 127.0.0.1:37092)
0810 11:35:40.615156 (+ 2169us) negotiation.cc:107] Waiting for socket to connect
0810 11:35:40.615174 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 11:35:40.615365 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:35:40.616507 (+ 1142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:35:40.616511 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:35:40.616552 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:35:40.617279 (+ 727us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:35:40.617302 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:35:40.619415 (+ 2113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:35:40.619421 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:35:40.621890 (+ 2469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:35:40.621905 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:35:40.624156 (+ 2251us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:35:40.624165 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:35:40.624327 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:35:40.624361 (+ 34us) client_negotiation.cc:769] Sending connection context
0810 11:35:40.624445 (+ 84us) client_negotiation.cc:240] Negotiation successful
0810 11:35:40.624607 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1643,"mutex_wait_us":41,"spinlock_wait_cycles":31616,"thread_start_us":1359,"threads_started":1}
I20260810 11:35:40.625167 28059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:40.613743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37092 (local address 127.8.18.195:39803)
0810 11:35:40.615056 (+ 1313us) server_negotiation.cc:206] Beginning negotiation
0810 11:35:40.615065 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 11:35:40.615590 (+ 525us) server_negotiation.cc:407] Connection header received
0810 11:35:40.615712 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:35:40.615722 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:35:40.615995 (+ 273us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:35:40.616182 (+ 187us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:35:40.617562 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:40.619242 (+ 1680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:40.622118 (+ 2876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:40.623543 (+ 1425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:40.623702 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:35:40.623976 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0810 11:35:40.624612 (+ 636us) server_negotiation.cc:299] Negotiation successful
0810 11:35:40.624800 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1125,"thread_start_us":962,"threads_started":1}
I20260810 11:35:40.625553 28058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:40.611689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.193:43757 (local address 127.0.0.1:48312)
0810 11:35:40.615156 (+ 3467us) negotiation.cc:107] Waiting for socket to connect
0810 11:35:40.615174 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 11:35:40.615365 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:35:40.616496 (+ 1131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:35:40.616506 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:35:40.616549 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:35:40.617337 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:35:40.617354 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:35:40.618810 (+ 1456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:35:40.618818 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:35:40.621060 (+ 2242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:35:40.621074 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:35:40.622897 (+ 1823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:35:40.622903 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:35:40.623112 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:35:40.623180 (+ 68us) client_negotiation.cc:769] Sending connection context
0810 11:35:40.623321 (+ 141us) client_negotiation.cc:240] Negotiation successful
0810 11:35:40.624986 (+ 1665us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2846,"mutex_wait_us":106,"spinlock_wait_cycles":3456,"thread_start_us":2770,"threads_started":1}
I20260810 11:35:40.626245 28056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:40.611754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33852 (local address 127.8.18.194:36477)
0810 11:35:40.613346 (+ 1592us) server_negotiation.cc:206] Beginning negotiation
0810 11:35:40.613358 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 11:35:40.615040 (+ 1682us) server_negotiation.cc:407] Connection header received
0810 11:35:40.615789 (+ 749us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:35:40.615796 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:35:40.615999 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:35:40.616179 (+ 180us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:35:40.617701 (+ 1522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:40.619540 (+ 1839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:40.622602 (+ 3062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:40.624056 (+ 1454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:40.624217 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:35:40.624551 (+ 334us) server_negotiation.cc:1035] Waiting for connection context
0810 11:35:40.625483 (+ 932us) server_negotiation.cc:299] Negotiation successful
0810 11:35:40.625709 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1370,"thread_start_us":1253,"threads_started":1}
I20260810 11:35:40.626833 28054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:40.611632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.194:36477 (local address 127.0.0.1:33852)
0810 11:35:40.614844 (+ 3212us) negotiation.cc:107] Waiting for socket to connect
0810 11:35:40.614894 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0810 11:35:40.615583 (+ 689us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:35:40.616647 (+ 1064us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:35:40.616654 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:35:40.616680 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:35:40.617389 (+ 709us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:35:40.617401 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:35:40.620143 (+ 2742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:35:40.620148 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:35:40.622406 (+ 2258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:35:40.622417 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:35:40.624706 (+ 2289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:35:40.624712 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:35:40.624893 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:35:40.624934 (+ 41us) client_negotiation.cc:769] Sending connection context
0810 11:35:40.625024 (+ 90us) client_negotiation.cc:240] Negotiation successful
0810 11:35:40.625173 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2237,"mutex_wait_us":65,"thread_start_us":1905,"threads_started":1}
I20260810 11:35:40.629880 27975 tablet_service.cc:1467] Processing CreateTablet for tablet fb3af83721f54bbeaed3cb79f8a1c8af (DEFAULT_TABLE table=test-table [id=32ebd2f7ced748e2b51fa46c15814e6f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:35:40.629884 27704 tablet_service.cc:1467] Processing CreateTablet for tablet fb3af83721f54bbeaed3cb79f8a1c8af (DEFAULT_TABLE table=test-table [id=32ebd2f7ced748e2b51fa46c15814e6f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:35:40.630293 27840 tablet_service.cc:1467] Processing CreateTablet for tablet fb3af83721f54bbeaed3cb79f8a1c8af (DEFAULT_TABLE table=test-table [id=32ebd2f7ced748e2b51fa46c15814e6f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:35:40.631537 27840 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fb3af83721f54bbeaed3cb79f8a1c8af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:35:40.631562 27975 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fb3af83721f54bbeaed3cb79f8a1c8af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:35:40.631559 27704 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fb3af83721f54bbeaed3cb79f8a1c8af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:35:40.647539 28061 tablet_bootstrap.cc:492] T fb3af83721f54bbeaed3cb79f8a1c8af P 7f5a9229ded94ae98545144a2859f2e5: Bootstrap starting.
I20260810 11:35:40.648664 28062 tablet_bootstrap.cc:492] T fb3af83721f54bbeaed3cb79f8a1c8af P df43fc9333754a9db8905fd393d4aa31: Bootstrap starting.
I20260810 11:35:40.650044 28060 tablet_bootstrap.cc:492] T fb3af83721f54bbeaed3cb79f8a1c8af P 918a4750cfb44a89ac206d5a43bb91d1: Bootstrap starting.
I20260810 11:35:40.653824 28060 tablet_bootstrap.cc:654] T fb3af83721f54bbeaed3cb79f8a1c8af P 918a4750cfb44a89ac206d5a43bb91d1: Neither blocks nor log segments found. Creating new log.
I20260810 11:35:40.654078 28061 tablet_bootstrap.cc:654] T fb3af83721f54bbeaed3cb79f8a1c8af P 7f5a9229ded94ae98545144a2859f2e5: Neither blocks nor log segments found. Creating new log.
I20260810 11:35:40.654079 28062 tablet_bootstrap.cc:654] T fb3af83721f54bbeaed3cb79f8a1c8af P df43fc9333754a9db8905fd393d4aa31: Neither blocks nor log segments found. Creating new log.
I20260810 11:35:40.655068 28060 log.cc:824] T fb3af83721f54bbeaed3cb79f8a1c8af P 918a4750cfb44a89ac206d5a43bb91d1: Log is configured to *not* fsync() on all Append() calls
I20260810 11:35:40.655359 28061 log.cc:824] T fb3af83721f54bbeaed3cb79f8a1c8af P 7f5a9229ded94ae98545144a2859f2e5: Log is configured to *not* fsync() on all Append() calls
I20260810 11:35:40.655365 28062 log.cc:824] T fb3af83721f54bbeaed3cb79f8a1c8af P df43fc9333754a9db8905fd393d4aa31: Log is configured to *not* fsync() on all Append() calls
I20260810 11:35:40.659044 28062 tablet_bootstrap.cc:492] T fb3af83721f54bbeaed3cb79f8a1c8af P df43fc9333754a9db8905fd393d4aa31: No bootstrap required, opened a new log
I20260810 11:35:40.659037 28061 tablet_bootstrap.cc:492] T fb3af83721f54bbeaed3cb79f8a1c8af P 7f5a9229ded94ae98545144a2859f2e5: No bootstrap required, opened a new log
I20260810 11:35:40.659040 28060 tablet_bootstrap.cc:492] T fb3af83721f54bbeaed3cb79f8a1c8af P 918a4750cfb44a89ac206d5a43bb91d1: No bootstrap required, opened a new log
I20260810 11:35:40.659322 28060 ts_tablet_manager.cc:1397] T fb3af83721f54bbeaed3cb79f8a1c8af P 918a4750cfb44a89ac206d5a43bb91d1: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260810 11:35:40.659339 28061 ts_tablet_manager.cc:1397] T fb3af83721f54bbeaed3cb79f8a1c8af P 7f5a9229ded94ae98545144a2859f2e5: Time spent bootstrapping tablet: real 0.012s user 0.007s sys 0.000s
I20260810 11:35:40.659340 28062 ts_tablet_manager.cc:1397] T fb3af83721f54bbeaed3cb79f8a1c8af P df43fc9333754a9db8905fd393d4aa31: Time spent bootstrapping tablet: real 0.011s user 0.009s sys 0.000s
I20260810 11:35:40.667591 28060 raft_consensus.cc:348] T fb3af83721f54bbeaed3cb79f8a1c8af P 918a4750cfb44a89ac206d5a43bb91d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918a4750cfb44a89ac206d5a43bb91d1" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 36477 } } peers { permanent_uuid: "7f5a9229ded94ae98545144a2859f2e5" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 43757 } } peers { permanent_uuid: "df43fc9333754a9db8905fd393d4aa31" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 39803 } }
I20260810 11:35:40.668062 28060 raft_consensus.cc:374] T fb3af83721f54bbeaed3cb79f8a1c8af P 918a4750cfb44a89ac206d5a43bb91d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:35:40.668221 28060 raft_consensus.cc:729] T fb3af83721f54bbeaed3cb79f8a1c8af P 918a4750cfb44a89ac206d5a43bb91d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 918a4750cfb44a89ac206d5a43bb91d1, State: Initialized, Role: FOLLOWER
I20260810 11:35:40.668939 28060 consensus_queue.cc:260] T fb3af83721f54bbeaed3cb79f8a1c8af P 918a4750cfb44a89ac206d5a43bb91d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918a4750cfb44a89ac206d5a43bb91d1" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 36477 } } peers { permanent_uuid: "7f5a9229ded94ae98545144a2859f2e5" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 43757 } } peers { permanent_uuid: "df43fc9333754a9db8905fd393d4aa31" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 39803 } }
I20260810 11:35:40.670225 27906 heartbeater.cc:503] Master 127.8.18.254:35041 was elected leader, sending a full tablet report...
I20260810 11:35:40.671027 28060 ts_tablet_manager.cc:1428] T fb3af83721f54bbeaed3cb79f8a1c8af P 918a4750cfb44a89ac206d5a43bb91d1: Time spent starting tablet: real 0.012s user 0.011s sys 0.001s
I20260810 11:35:40.670698 28062 raft_consensus.cc:348] T fb3af83721f54bbeaed3cb79f8a1c8af P df43fc9333754a9db8905fd393d4aa31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918a4750cfb44a89ac206d5a43bb91d1" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 36477 } } peers { permanent_uuid: "7f5a9229ded94ae98545144a2859f2e5" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 43757 } } peers { permanent_uuid: "df43fc9333754a9db8905fd393d4aa31" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 39803 } }
I20260810 11:35:40.670697 28061 raft_consensus.cc:348] T fb3af83721f54bbeaed3cb79f8a1c8af P 7f5a9229ded94ae98545144a2859f2e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918a4750cfb44a89ac206d5a43bb91d1" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 36477 } } peers { permanent_uuid: "7f5a9229ded94ae98545144a2859f2e5" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 43757 } } peers { permanent_uuid: "df43fc9333754a9db8905fd393d4aa31" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 39803 } }
I20260810 11:35:40.671193 28061 raft_consensus.cc:374] T fb3af83721f54bbeaed3cb79f8a1c8af P 7f5a9229ded94ae98545144a2859f2e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:35:40.671187 28062 raft_consensus.cc:374] T fb3af83721f54bbeaed3cb79f8a1c8af P df43fc9333754a9db8905fd393d4aa31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:35:40.671290 28061 raft_consensus.cc:729] T fb3af83721f54bbeaed3cb79f8a1c8af P 7f5a9229ded94ae98545144a2859f2e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f5a9229ded94ae98545144a2859f2e5, State: Initialized, Role: FOLLOWER
I20260810 11:35:40.671290 28062 raft_consensus.cc:729] T fb3af83721f54bbeaed3cb79f8a1c8af P df43fc9333754a9db8905fd393d4aa31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df43fc9333754a9db8905fd393d4aa31, State: Initialized, Role: FOLLOWER
I20260810 11:35:40.672075 28062 consensus_queue.cc:260] T fb3af83721f54bbeaed3cb79f8a1c8af P df43fc9333754a9db8905fd393d4aa31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918a4750cfb44a89ac206d5a43bb91d1" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 36477 } } peers { permanent_uuid: "7f5a9229ded94ae98545144a2859f2e5" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 43757 } } peers { permanent_uuid: "df43fc9333754a9db8905fd393d4aa31" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 39803 } }
I20260810 11:35:40.672077 28061 consensus_queue.cc:260] T fb3af83721f54bbeaed3cb79f8a1c8af P 7f5a9229ded94ae98545144a2859f2e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918a4750cfb44a89ac206d5a43bb91d1" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 36477 } } peers { permanent_uuid: "7f5a9229ded94ae98545144a2859f2e5" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 43757 } } peers { permanent_uuid: "df43fc9333754a9db8905fd393d4aa31" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 39803 } }
I20260810 11:35:40.673942 28042 heartbeater.cc:503] Master 127.8.18.254:35041 was elected leader, sending a full tablet report...
I20260810 11:35:40.674602 28062 ts_tablet_manager.cc:1428] T fb3af83721f54bbeaed3cb79f8a1c8af P df43fc9333754a9db8905fd393d4aa31: Time spent starting tablet: real 0.015s user 0.015s sys 0.000s
I20260810 11:35:40.674628 28061 ts_tablet_manager.cc:1428] T fb3af83721f54bbeaed3cb79f8a1c8af P 7f5a9229ded94ae98545144a2859f2e5: Time spent starting tablet: real 0.015s user 0.016s sys 0.000s
W20260810 11:35:40.773497 27773 tablet.cc:2369] T fb3af83721f54bbeaed3cb79f8a1c8af P 7f5a9229ded94ae98545144a2859f2e5: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 11:35:40.791712 27907 tablet.cc:2369] T fb3af83721f54bbeaed3cb79f8a1c8af P 918a4750cfb44a89ac206d5a43bb91d1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 11:35:40.795642 28069 raft_consensus.cc:482] T fb3af83721f54bbeaed3cb79f8a1c8af P 7f5a9229ded94ae98545144a2859f2e5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 11:35:40.795969 28069 raft_consensus.cc:504] T fb3af83721f54bbeaed3cb79f8a1c8af P 7f5a9229ded94ae98545144a2859f2e5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918a4750cfb44a89ac206d5a43bb91d1" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 36477 } } peers { permanent_uuid: "7f5a9229ded94ae98545144a2859f2e5" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 43757 } } peers { permanent_uuid: "df43fc9333754a9db8905fd393d4aa31" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 39803 } }
I20260810 11:35:40.797808 28069 leader_election.cc:288] T fb3af83721f54bbeaed3cb79f8a1c8af P 7f5a9229ded94ae98545144a2859f2e5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 918a4750cfb44a89ac206d5a43bb91d1 (127.8.18.194:36477), df43fc9333754a9db8905fd393d4aa31 (127.8.18.195:39803)
W20260810 11:35:40.803208 28043 tablet.cc:2369] T fb3af83721f54bbeaed3cb79f8a1c8af P df43fc9333754a9db8905fd393d4aa31: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 11:35:40.808079 28070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:40.798241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.195:39803 (local address 127.8.18.193:37063)
0810 11:35:40.799356 (+ 1115us) negotiation.cc:107] Waiting for socket to connect
0810 11:35:40.799372 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 11:35:40.799533 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:35:40.799926 (+ 393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:35:40.799931 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:35:40.799979 (+ 48us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:35:40.800554 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:35:40.800575 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:35:40.802304 (+ 1729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:35:40.802316 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:35:40.805309 (+ 2993us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:35:40.805332 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:35:40.807172 (+ 1840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:35:40.807179 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:35:40.807347 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:35:40.807389 (+ 42us) client_negotiation.cc:769] Sending connection context
0810 11:35:40.807500 (+ 111us) client_negotiation.cc:240] Negotiation successful
0810 11:35:40.807662 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1002,"mutex_wait_us":291,"spinlock_wait_cycles":47616,"thread_start_us":607,"threads_started":1}
I20260810 11:35:40.808207 28056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:40.798430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.193:51261 (local address 127.8.18.194:36477)
0810 11:35:40.798640 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0810 11:35:40.798647 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:35:40.799466 (+ 819us) server_negotiation.cc:407] Connection header received
0810 11:35:40.799681 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:35:40.799688 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:35:40.799788 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:35:40.799887 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:35:40.800947 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:40.802121 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:40.805637 (+ 3516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:40.806936 (+ 1299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:40.807114 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:35:40.807191 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0810 11:35:40.807793 (+ 602us) server_negotiation.cc:299] Negotiation successful
0810 11:35:40.807941 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260810 11:35:40.808451 28059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:40.798437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.193:37063 (local address 127.8.18.195:39803)
0810 11:35:40.798640 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0810 11:35:40.798647 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:35:40.799475 (+ 828us) server_negotiation.cc:407] Connection header received
0810 11:35:40.799682 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:35:40.799688 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:35:40.799775 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:35:40.799887 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:35:40.800898 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:40.802132 (+ 1234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:40.805634 (+ 3502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:40.806936 (+ 1302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:40.807119 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:35:40.807236 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0810 11:35:40.807571 (+ 335us) server_negotiation.cc:299] Negotiation successful
0810 11:35:40.807739 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260810 11:35:40.809214 28071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:40.798172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.194:36477 (local address 127.8.18.193:51261)
0810 11:35:40.799287 (+ 1115us) negotiation.cc:107] Waiting for socket to connect
0810 11:35:40.799308 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 11:35:40.799492 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:35:40.799926 (+ 434us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:35:40.799931 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:35:40.799977 (+ 46us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:35:40.800614 (+ 637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:35:40.800628 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:35:40.802304 (+ 1676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:35:40.802310 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:35:40.805309 (+ 2999us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:35:40.805333 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:35:40.807373 (+ 2040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:35:40.807378 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:35:40.807562 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:35:40.807598 (+ 36us) client_negotiation.cc:769] Sending connection context
0810 11:35:40.808698 (+ 1100us) client_negotiation.cc:240] Negotiation successful
0810 11:35:40.808864 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":942,"mutex_wait_us":258,"spinlock_wait_cycles":16000,"thread_start_us":714,"threads_started":1}
I20260810 11:35:40.809564 27997 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb3af83721f54bbeaed3cb79f8a1c8af" candidate_uuid: "7f5a9229ded94ae98545144a2859f2e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df43fc9333754a9db8905fd393d4aa31" is_pre_election: true
I20260810 11:35:40.810190 27997 raft_consensus.cc:2454] T fb3af83721f54bbeaed3cb79f8a1c8af P df43fc9333754a9db8905fd393d4aa31 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7f5a9229ded94ae98545144a2859f2e5 in term 0.
I20260810 11:35:40.810371 27860 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb3af83721f54bbeaed3cb79f8a1c8af" candidate_uuid: "7f5a9229ded94ae98545144a2859f2e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "918a4750cfb44a89ac206d5a43bb91d1" is_pre_election: true
I20260810 11:35:40.811060 27860 raft_consensus.cc:2454] T fb3af83721f54bbeaed3cb79f8a1c8af P 918a4750cfb44a89ac206d5a43bb91d1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7f5a9229ded94ae98545144a2859f2e5 in term 0.
I20260810 11:35:40.811182 27658 leader_election.cc:302] T fb3af83721f54bbeaed3cb79f8a1c8af P 7f5a9229ded94ae98545144a2859f2e5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7f5a9229ded94ae98545144a2859f2e5, df43fc9333754a9db8905fd393d4aa31; no voters:
I20260810 11:35:40.811744 28069 raft_consensus.cc:2781] T fb3af83721f54bbeaed3cb79f8a1c8af P 7f5a9229ded94ae98545144a2859f2e5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 11:35:40.811915 28069 raft_consensus.cc:482] T fb3af83721f54bbeaed3cb79f8a1c8af P 7f5a9229ded94ae98545144a2859f2e5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 11:35:40.812045 28069 raft_consensus.cc:3037] T fb3af83721f54bbeaed3cb79f8a1c8af P 7f5a9229ded94ae98545144a2859f2e5 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:35:40.814548 28069 raft_consensus.cc:504] T fb3af83721f54bbeaed3cb79f8a1c8af P 7f5a9229ded94ae98545144a2859f2e5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918a4750cfb44a89ac206d5a43bb91d1" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 36477 } } peers { permanent_uuid: "7f5a9229ded94ae98545144a2859f2e5" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 43757 } } peers { permanent_uuid: "df43fc9333754a9db8905fd393d4aa31" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 39803 } }
I20260810 11:35:40.815331 28069 leader_election.cc:288] T fb3af83721f54bbeaed3cb79f8a1c8af P 7f5a9229ded94ae98545144a2859f2e5 [CANDIDATE]: Term 1 election: Requested vote from peers 918a4750cfb44a89ac206d5a43bb91d1 (127.8.18.194:36477), df43fc9333754a9db8905fd393d4aa31 (127.8.18.195:39803)
I20260810 11:35:40.815979 27860 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb3af83721f54bbeaed3cb79f8a1c8af" candidate_uuid: "7f5a9229ded94ae98545144a2859f2e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "918a4750cfb44a89ac206d5a43bb91d1"
I20260810 11:35:40.816056 27997 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb3af83721f54bbeaed3cb79f8a1c8af" candidate_uuid: "7f5a9229ded94ae98545144a2859f2e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df43fc9333754a9db8905fd393d4aa31"
I20260810 11:35:40.816269 27860 raft_consensus.cc:3037] T fb3af83721f54bbeaed3cb79f8a1c8af P 918a4750cfb44a89ac206d5a43bb91d1 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:35:40.816269 27997 raft_consensus.cc:3037] T fb3af83721f54bbeaed3cb79f8a1c8af P df43fc9333754a9db8905fd393d4aa31 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:35:40.819864 27997 raft_consensus.cc:2454] T fb3af83721f54bbeaed3cb79f8a1c8af P df43fc9333754a9db8905fd393d4aa31 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7f5a9229ded94ae98545144a2859f2e5 in term 1.
I20260810 11:35:40.819864 27860 raft_consensus.cc:2454] T fb3af83721f54bbeaed3cb79f8a1c8af P 918a4750cfb44a89ac206d5a43bb91d1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7f5a9229ded94ae98545144a2859f2e5 in term 1.
I20260810 11:35:40.820601 27655 leader_election.cc:302] T fb3af83721f54bbeaed3cb79f8a1c8af P 7f5a9229ded94ae98545144a2859f2e5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 7f5a9229ded94ae98545144a2859f2e5, 918a4750cfb44a89ac206d5a43bb91d1, df43fc9333754a9db8905fd393d4aa31; no voters:
I20260810 11:35:40.821024 28069 raft_consensus.cc:2781] T fb3af83721f54bbeaed3cb79f8a1c8af P 7f5a9229ded94ae98545144a2859f2e5 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:35:40.821656 28069 raft_consensus.cc:686] T fb3af83721f54bbeaed3cb79f8a1c8af P 7f5a9229ded94ae98545144a2859f2e5 [term 1 LEADER]: Becoming Leader. State: Replica: 7f5a9229ded94ae98545144a2859f2e5, State: Running, Role: LEADER
I20260810 11:35:40.822219 28069 consensus_queue.cc:237] T fb3af83721f54bbeaed3cb79f8a1c8af P 7f5a9229ded94ae98545144a2859f2e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918a4750cfb44a89ac206d5a43bb91d1" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 36477 } } peers { permanent_uuid: "7f5a9229ded94ae98545144a2859f2e5" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 43757 } } peers { permanent_uuid: "df43fc9333754a9db8905fd393d4aa31" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 39803 } }
I20260810 11:35:40.827576 27577 catalog_manager.cc:5495] T fb3af83721f54bbeaed3cb79f8a1c8af P 7f5a9229ded94ae98545144a2859f2e5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7f5a9229ded94ae98545144a2859f2e5 (127.8.18.193). New cstate: current_term: 1 leader_uuid: "7f5a9229ded94ae98545144a2859f2e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "918a4750cfb44a89ac206d5a43bb91d1" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 36477 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7f5a9229ded94ae98545144a2859f2e5" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 43757 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "df43fc9333754a9db8905fd393d4aa31" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 39803 } health_report { overall_health: UNKNOWN } } }
I20260810 11:35:40.867425 28057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:40.861598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48314 (local address 127.8.18.193:43757)
0810 11:35:40.861848 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0810 11:35:40.861853 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:35:40.861872 (+ 19us) server_negotiation.cc:407] Connection header received
0810 11:35:40.862044 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:35:40.862048 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:35:40.862187 (+ 139us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:35:40.862326 (+ 139us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 11:35:40.862706 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:40.863449 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:40.864919 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:40.865382 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:40.865565 (+ 183us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:35:40.866114 (+ 549us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 11:35:40.866668 (+ 554us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 11:35:40.866806 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0810 11:35:40.866987 (+ 181us) server_negotiation.cc:299] Negotiation successful
0810 11:35:40.867071 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260810 11:35:40.877359 27997 raft_consensus.cc:1261] T fb3af83721f54bbeaed3cb79f8a1c8af P df43fc9333754a9db8905fd393d4aa31 [term 1 FOLLOWER]: Refusing update from remote peer 7f5a9229ded94ae98545144a2859f2e5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 11:35:40.877359 27860 raft_consensus.cc:1261] T fb3af83721f54bbeaed3cb79f8a1c8af P 918a4750cfb44a89ac206d5a43bb91d1 [term 1 FOLLOWER]: Refusing update from remote peer 7f5a9229ded94ae98545144a2859f2e5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 11:35:40.878692 28069 consensus_queue.cc:1035] T fb3af83721f54bbeaed3cb79f8a1c8af P 7f5a9229ded94ae98545144a2859f2e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "df43fc9333754a9db8905fd393d4aa31" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 39803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 11:35:40.879524 28069 consensus_queue.cc:1035] T fb3af83721f54bbeaed3cb79f8a1c8af P 7f5a9229ded94ae98545144a2859f2e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "918a4750cfb44a89ac206d5a43bb91d1" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 36477 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 11:35:40.891611 28076 mvcc.cc:205] Tried to move back new op lower bound from 7316937690620239872 to 7316937690411991040. Current Snapshot: MvccSnapshot[applied={T|T < 7316937690620239872}]
I20260810 11:35:40.892299 28077 mvcc.cc:205] Tried to move back new op lower bound from 7316937690620239872 to 7316937690411991040. Current Snapshot: MvccSnapshot[applied={T|T < 7316937690620239872}]
I20260810 11:35:40.913264 27577 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43720:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 11:35:40.913642 27577 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43720:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 11:35:40.917927 27577 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d46b9d4a638641f3a59dac2a63c23b45: Sending DeleteTablet for 3 replicas of tablet fb3af83721f54bbeaed3cb79f8a1c8af
I20260810 11:35:40.919350 27840 tablet_service.cc:1514] Processing DeleteTablet for tablet fb3af83721f54bbeaed3cb79f8a1c8af with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:35:40 UTC) from {username='slave'} at 127.0.0.1:33852
I20260810 11:35:40.919409 27975 tablet_service.cc:1514] Processing DeleteTablet for tablet fb3af83721f54bbeaed3cb79f8a1c8af with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:35:40 UTC) from {username='slave'} at 127.0.0.1:37092
I20260810 11:35:40.920609 27704 tablet_service.cc:1514] Processing DeleteTablet for tablet fb3af83721f54bbeaed3cb79f8a1c8af with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:35:40 UTC) from {username='slave'} at 127.0.0.1:48312
I20260810 11:35:40.922885 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 27639
I20260810 11:35:40.930187 28087 tablet_replica.cc:330] T fb3af83721f54bbeaed3cb79f8a1c8af P 7f5a9229ded94ae98545144a2859f2e5: stopping tablet replica
I20260810 11:35:40.930878 28087 raft_consensus.cc:2229] T fb3af83721f54bbeaed3cb79f8a1c8af P 7f5a9229ded94ae98545144a2859f2e5 [term 1 LEADER]: Raft consensus shutting down.
I20260810 11:35:40.931809 28087 raft_consensus.cc:2258] T fb3af83721f54bbeaed3cb79f8a1c8af P 7f5a9229ded94ae98545144a2859f2e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:35:40.934545 28085 tablet_replica.cc:330] T fb3af83721f54bbeaed3cb79f8a1c8af P 918a4750cfb44a89ac206d5a43bb91d1: stopping tablet replica
I20260810 11:35:40.935307 28086 tablet_replica.cc:330] T fb3af83721f54bbeaed3cb79f8a1c8af P df43fc9333754a9db8905fd393d4aa31: stopping tablet replica
I20260810 11:35:40.935978 28086 raft_consensus.cc:2229] T fb3af83721f54bbeaed3cb79f8a1c8af P df43fc9333754a9db8905fd393d4aa31 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 11:35:40.936533 28085 raft_consensus.cc:2229] T fb3af83721f54bbeaed3cb79f8a1c8af P 918a4750cfb44a89ac206d5a43bb91d1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 11:35:40.937213 28086 raft_consensus.cc:2258] T fb3af83721f54bbeaed3cb79f8a1c8af P df43fc9333754a9db8905fd393d4aa31 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:35:40.937755 28087 ts_tablet_manager.cc:1905] T fb3af83721f54bbeaed3cb79f8a1c8af P 7f5a9229ded94ae98545144a2859f2e5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:35:40.937775 28085 raft_consensus.cc:2258] T fb3af83721f54bbeaed3cb79f8a1c8af P 918a4750cfb44a89ac206d5a43bb91d1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:35:40.940326 28085 ts_tablet_manager.cc:1905] T fb3af83721f54bbeaed3cb79f8a1c8af P 918a4750cfb44a89ac206d5a43bb91d1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:35:40.942059 28086 ts_tablet_manager.cc:1905] T fb3af83721f54bbeaed3cb79f8a1c8af P df43fc9333754a9db8905fd393d4aa31: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:35:40.946758 28087 ts_tablet_manager.cc:1918] T fb3af83721f54bbeaed3cb79f8a1c8af P 7f5a9229ded94ae98545144a2859f2e5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 11:35:40.947307 28087 log.cc:1196] T fb3af83721f54bbeaed3cb79f8a1c8af P 7f5a9229ded94ae98545144a2859f2e5: Deleting WAL directory at /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786361401922362-8267-0/minicluster-data/ts-0/wal/wals/fb3af83721f54bbeaed3cb79f8a1c8af
I20260810 11:35:40.948063 28087 ts_tablet_manager.cc:1939] T fb3af83721f54bbeaed3cb79f8a1c8af P 7f5a9229ded94ae98545144a2859f2e5: Deleting consensus metadata
I20260810 11:35:40.950086 28086 ts_tablet_manager.cc:1918] T fb3af83721f54bbeaed3cb79f8a1c8af P df43fc9333754a9db8905fd393d4aa31: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 11:35:40.950083 28085 ts_tablet_manager.cc:1918] T fb3af83721f54bbeaed3cb79f8a1c8af P 918a4750cfb44a89ac206d5a43bb91d1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 11:35:40.950282 28085 log.cc:1196] T fb3af83721f54bbeaed3cb79f8a1c8af P 918a4750cfb44a89ac206d5a43bb91d1: Deleting WAL directory at /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786361401922362-8267-0/minicluster-data/ts-1/wal/wals/fb3af83721f54bbeaed3cb79f8a1c8af
I20260810 11:35:40.950455 27564 catalog_manager.cc:4841] TS 7f5a9229ded94ae98545144a2859f2e5 (127.8.18.193:43757): tablet fb3af83721f54bbeaed3cb79f8a1c8af (table test-table [id=32ebd2f7ced748e2b51fa46c15814e6f]) successfully deleted
I20260810 11:35:40.950986 28085 ts_tablet_manager.cc:1939] T fb3af83721f54bbeaed3cb79f8a1c8af P 918a4750cfb44a89ac206d5a43bb91d1: Deleting consensus metadata
I20260810 11:35:40.953025 28057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:40.924637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48326 (local address 127.8.18.193:43757)
0810 11:35:40.924833 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0810 11:35:40.924839 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:35:40.935592 (+ 10753us) server_negotiation.cc:407] Connection header received
0810 11:35:40.935759 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:35:40.935767 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:35:40.935878 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:35:40.936020 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:35:40.936958 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:40.938175 (+ 1217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:40.940924 (+ 2749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:40.941697 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:40.941804 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:35:40.950945 (+ 9141us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:35:40.950962 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:35:40.950984 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:35:40.951931 (+ 947us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:35:40.952011 (+ 80us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:35:40.952043 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:35:40.952141 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0810 11:35:40.952501 (+ 360us) server_negotiation.cc:299] Negotiation successful
0810 11:35:40.952641 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260810 11:35:40.953058 28086 log.cc:1196] T fb3af83721f54bbeaed3cb79f8a1c8af P df43fc9333754a9db8905fd393d4aa31: Deleting WAL directory at /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786361401922362-8267-0/minicluster-data/ts-2/wal/wals/fb3af83721f54bbeaed3cb79f8a1c8af
I20260810 11:35:40.953020 27562 catalog_manager.cc:4841] TS 918a4750cfb44a89ac206d5a43bb91d1 (127.8.18.194:36477): tablet fb3af83721f54bbeaed3cb79f8a1c8af (table test-table [id=32ebd2f7ced748e2b51fa46c15814e6f]) successfully deleted
I20260810 11:35:40.953814 28086 ts_tablet_manager.cc:1939] T fb3af83721f54bbeaed3cb79f8a1c8af P df43fc9333754a9db8905fd393d4aa31: Deleting consensus metadata
I20260810 11:35:40.954210 27767 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:35:40.956041 27565 catalog_manager.cc:4841] TS df43fc9333754a9db8905fd393d4aa31 (127.8.18.195:39803): tablet fb3af83721f54bbeaed3cb79f8a1c8af (table test-table [id=32ebd2f7ced748e2b51fa46c15814e6f]) successfully deleted
I20260810 11:35:41.067121 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 27639
I20260810 11:35:41.086652 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 27776
I20260810 11:35:41.094460 28056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:41.087597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33866 (local address 127.8.18.194:36477)
0810 11:35:41.087867 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0810 11:35:41.087872 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:35:41.087893 (+ 21us) server_negotiation.cc:407] Connection header received
0810 11:35:41.088063 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:35:41.088068 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:35:41.088187 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:35:41.088304 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:35:41.088736 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:41.089600 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:41.090953 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:41.091394 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:41.091550 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:35:41.092759 (+ 1209us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:35:41.092769 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:35:41.092785 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:35:41.093565 (+ 780us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:35:41.093640 (+ 75us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:35:41.093670 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:35:41.093797 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0810 11:35:41.093946 (+ 149us) server_negotiation.cc:299] Negotiation successful
0810 11:35:41.094071 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109}
I20260810 11:35:41.095214 27901 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:35:41.188491 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 27776
I20260810 11:35:41.209029 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 27910
I20260810 11:35:41.216539 28059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:41.209976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37098 (local address 127.8.18.195:39803)
0810 11:35:41.210181 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0810 11:35:41.210190 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 11:35:41.210259 (+ 69us) server_negotiation.cc:407] Connection header received
0810 11:35:41.210461 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:35:41.210467 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:35:41.210567 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:35:41.210694 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:35:41.211147 (+ 453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:41.211905 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:41.213235 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:41.213731 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:41.213878 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:35:41.215113 (+ 1235us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:35:41.215125 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:35:41.215141 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:35:41.215722 (+ 581us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:35:41.215777 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:35:41.215797 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:35:41.215931 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0810 11:35:41.216070 (+ 139us) server_negotiation.cc:299] Negotiation successful
0810 11:35:41.216195 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260810 11:35:41.217373 28037 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:35:41.309921 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 27910
I20260810 11:35:41.331530 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 27547
I20260810 11:35:41.332914 27610 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:35:41.450204 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 27547
2026-08-10T11:35:41Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (3156 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786361401922362-8267-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 11:35:41 dist-test-slave-srgv krb5kdc[28096](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 11:35:41 dist-test-slave-srgv krb5kdc[28096](info): set up 2 sockets
Aug 10 11:35:41 dist-test-slave-srgv krb5kdc[28096](info): commencing operation
krb5kdc: starting...
W20260810 11:35:42.616851 8267 mini_kdc.cc:121] Time spent starting KDC: real 1.119s user 0.000s sys 0.041s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786361401922362-8267-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786361401922362-8267-0/krb5kdc/test-user.keytab.
Aug 10 11:35:42 dist-test-slave-srgv krb5kdc[28096](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361742, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T11:35:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T11:35:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.18.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.18.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.18.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.254.keytab.
Entry for principal kudu/127.8.18.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.254.keytab.
I20260810 11:35:42.945603 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786361401922362-8267-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786361401922362-8267-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786361401922362-8267-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.18.254:38619
--webserver_interface=127.8.18.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.254.keytab
--principal=kudu/127.8.18.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.18.212:46507
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.18.254:38619
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786361401922362-8267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786361401922362-8267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786361401922362-8267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:35:43.316123 28109 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:35:43.316483 28109 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:35:43.316668 28109 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:35:43.325055 28109 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 11:35:43.325210 28109 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:35:43.325314 28109 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:35:43.325479 28109 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 11:35:43.325549 28109 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 11:35:43.337045 28109 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.18.212:46507
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786361401922362-8267-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786361401922362-8267-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.18.254:38619
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.254.keytab
--principal=kudu/127.8.18.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.18.254:38619
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.18.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786361401922362-8267-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:35:43.338678 28109 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:35:43.340169 28109 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:35:43.353647 28114 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:35:43.353735 28115 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:35:43.353713 28117 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:35:43.354722 28109 server_base.cc:1034] running on GCE node
I20260810 11:35:43.355721 28109 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:35:43.357349 28109 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:35:43.358651 28109 hybrid_clock.cc:648] HybridClock initialized: now 1786361743358576 us; error 67 us; skew 500 ppm
I20260810 11:35:43.359058 28109 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:35:43 dist-test-slave-srgv krb5kdc[28096](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361743, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:35:43.362993 28109 init.cc:377] Logged in from keytab as kudu/127.8.18.254@KRBTEST.COM (short username kudu)
I20260810 11:35:43.365018 28109 webserver.cc:460] Webserver started at http://127.8.18.254:38429/ using document root <none> and password file <none>
I20260810 11:35:43.365948 28109 fs_manager.cc:359] Metadata directory not provided
I20260810 11:35:43.366106 28109 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:35:43.366496 28109 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:35:43.370208 28109 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786361401922362-8267-0/minicluster-data/master-0/data/instance:
uuid: "342e92e798c6483a9561794ae1edb4bc"
format_stamp: "Formatted at 2026-08-10 11:35:43 on dist-test-slave-srgv"
server_key: "15d165c614095883bff4b39946984f24"
server_key_iv: "21c1344def7e6338d8068f6815c8f7d5"
server_key_version: "encryptionkey@0"
I20260810 11:35:43.371165 28109 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786361401922362-8267-0/minicluster-data/master-0/wal/instance:
uuid: "342e92e798c6483a9561794ae1edb4bc"
format_stamp: "Formatted at 2026-08-10 11:35:43 on dist-test-slave-srgv"
server_key: "15d165c614095883bff4b39946984f24"
server_key_iv: "21c1344def7e6338d8068f6815c8f7d5"
server_key_version: "encryptionkey@0"
I20260810 11:35:43.377282 28109 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.005s
I20260810 11:35:43.381829 28124 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:35:43.383608 28109 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260810 11:35:43.383869 28109 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786361401922362-8267-0/minicluster-data/master-0/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786361401922362-8267-0/minicluster-data/master-0/wal
uuid: "342e92e798c6483a9561794ae1edb4bc"
format_stamp: "Formatted at 2026-08-10 11:35:43 on dist-test-slave-srgv"
server_key: "15d165c614095883bff4b39946984f24"
server_key_iv: "21c1344def7e6338d8068f6815c8f7d5"
server_key_version: "encryptionkey@0"
I20260810 11:35:43.384179 28109 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786361401922362-8267-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786361401922362-8267-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786361401922362-8267-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:35:43.396665 28109 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:35:43.401790 28109 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:35:43.402154 28109 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:35:43.424594 28109 rpc_server.cc:307] RPC server started. Bound to: 127.8.18.254:38619
I20260810 11:35:43.424654 28175 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.18.254:38619 every 8 connection(s)
I20260810 11:35:43.426342 28109 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
I20260810 11:35:43.429713 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 28109
I20260810 11:35:43.429912 8267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786361401922362-8267-0/minicluster-data/master-0/wal/instance
I20260810 11:35:43.430445 8267 external_mini_cluster.cc:1428] Setting key 3ffb4fec3e2372a995de99b36cb2650e
I20260810 11:35:43.431128 28176 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:35:43.448056 28176 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 342e92e798c6483a9561794ae1edb4bc: Bootstrap starting.
Aug 10 11:35:43 dist-test-slave-srgv krb5kdc[28096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361742, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.18.254@KRBTEST.COM
I20260810 11:35:43.453222 28176 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 342e92e798c6483a9561794ae1edb4bc: Neither blocks nor log segments found. Creating new log.
I20260810 11:35:43.454998 28176 log.cc:824] T 00000000000000000000000000000000 P 342e92e798c6483a9561794ae1edb4bc: Log is configured to *not* fsync() on all Append() calls
I20260810 11:35:43.459429 28176 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 342e92e798c6483a9561794ae1edb4bc: No bootstrap required, opened a new log
I20260810 11:35:43.461509 28179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:43.432569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39264 (local address 127.8.18.254:38619)
0810 11:35:43.433549 (+ 980us) server_negotiation.cc:206] Beginning negotiation
0810 11:35:43.433562 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 11:35:43.433608 (+ 46us) server_negotiation.cc:407] Connection header received
0810 11:35:43.436077 (+ 2469us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:35:43.436104 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:35:43.436537 (+ 433us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:35:43.436996 (+ 459us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:35:43.439110 (+ 2114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:43.441162 (+ 2052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:43.443276 (+ 2114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:43.444089 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:43.444302 (+ 213us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:35:43.450445 (+ 6143us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:35:43.450460 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:35:43.450485 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:35:43.450564 (+ 79us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:35:43.455544 (+ 4980us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:35:43.456897 (+ 1353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:35:43.456909 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:35:43.456918 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:35:43.457201 (+ 283us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:35:43.457724 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:35:43.457729 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:35:43.457733 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:35:43.458530 (+ 797us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:35:43.458875 (+ 345us) server_negotiation.cc:1035] Waiting for connection context
0810 11:35:43.459368 (+ 493us) server_negotiation.cc:299] Negotiation successful
0810 11:35:43.459887 (+ 519us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":700,"thread_start_us":460,"threads_started":1}
I20260810 11:35:43.469317 28176 raft_consensus.cc:348] T 00000000000000000000000000000000 P 342e92e798c6483a9561794ae1edb4bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "342e92e798c6483a9561794ae1edb4bc" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 38619 } }
I20260810 11:35:43.469827 28176 raft_consensus.cc:374] T 00000000000000000000000000000000 P 342e92e798c6483a9561794ae1edb4bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:35:43.469913 28176 raft_consensus.cc:729] T 00000000000000000000000000000000 P 342e92e798c6483a9561794ae1edb4bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 342e92e798c6483a9561794ae1edb4bc, State: Initialized, Role: FOLLOWER
I20260810 11:35:43.470531 28176 consensus_queue.cc:260] T 00000000000000000000000000000000 P 342e92e798c6483a9561794ae1edb4bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "342e92e798c6483a9561794ae1edb4bc" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 38619 } }
I20260810 11:35:43.470788 28176 raft_consensus.cc:388] T 00000000000000000000000000000000 P 342e92e798c6483a9561794ae1edb4bc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 11:35:43.470944 28176 raft_consensus.cc:482] T 00000000000000000000000000000000 P 342e92e798c6483a9561794ae1edb4bc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 11:35:43.471164 28176 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 342e92e798c6483a9561794ae1edb4bc [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:35:43.473935 28176 raft_consensus.cc:504] T 00000000000000000000000000000000 P 342e92e798c6483a9561794ae1edb4bc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "342e92e798c6483a9561794ae1edb4bc" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 38619 } }
I20260810 11:35:43.474534 28176 leader_election.cc:302] T 00000000000000000000000000000000 P 342e92e798c6483a9561794ae1edb4bc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 342e92e798c6483a9561794ae1edb4bc; no voters:
I20260810 11:35:43.475193 28176 leader_election.cc:288] T 00000000000000000000000000000000 P 342e92e798c6483a9561794ae1edb4bc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 11:35:43.475379 28181 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 342e92e798c6483a9561794ae1edb4bc [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:35:43.476202 28181 raft_consensus.cc:686] T 00000000000000000000000000000000 P 342e92e798c6483a9561794ae1edb4bc [term 1 LEADER]: Becoming Leader. State: Replica: 342e92e798c6483a9561794ae1edb4bc, State: Running, Role: LEADER
I20260810 11:35:43.476856 28181 consensus_queue.cc:237] T 00000000000000000000000000000000 P 342e92e798c6483a9561794ae1edb4bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "342e92e798c6483a9561794ae1edb4bc" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 38619 } }
I20260810 11:35:43.477710 28176 sys_catalog.cc:564] T 00000000000000000000000000000000 P 342e92e798c6483a9561794ae1edb4bc [sys.catalog]: configured and running, proceeding with master startup.
I20260810 11:35:43.480619 28182 sys_catalog.cc:455] T 00000000000000000000000000000000 P 342e92e798c6483a9561794ae1edb4bc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "342e92e798c6483a9561794ae1edb4bc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "342e92e798c6483a9561794ae1edb4bc" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 38619 } } }
I20260810 11:35:43.480587 28183 sys_catalog.cc:455] T 00000000000000000000000000000000 P 342e92e798c6483a9561794ae1edb4bc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 342e92e798c6483a9561794ae1edb4bc. Latest consensus state: current_term: 1 leader_uuid: "342e92e798c6483a9561794ae1edb4bc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "342e92e798c6483a9561794ae1edb4bc" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 38619 } } }
I20260810 11:35:43.481113 28182 sys_catalog.cc:458] T 00000000000000000000000000000000 P 342e92e798c6483a9561794ae1edb4bc [sys.catalog]: This master's current role is: LEADER
I20260810 11:35:43.481128 28183 sys_catalog.cc:458] T 00000000000000000000000000000000 P 342e92e798c6483a9561794ae1edb4bc [sys.catalog]: This master's current role is: LEADER
I20260810 11:35:43.483673 28189 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 11:35:43.491597 28189 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 11:35:43.503067 28189 catalog_manager.cc:1347] Generated new cluster ID: 938a0f0d792b4f7bb1a1cd59e7c38581
I20260810 11:35:43.503224 28189 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 11:35:43.541049 28189 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 11:35:43.542650 28189 catalog_manager.cc:1504] Loading token signing keys...
I20260810 11:35:43.551510 28189 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 342e92e798c6483a9561794ae1edb4bc: Generated new TSK 0
I20260810 11:35:43.552465 28189 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.18.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.18.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.18.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.193.keytab.
Entry for principal kudu/127.8.18.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.193.keytab.
I20260810 11:35:43.633483 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786361401922362-8267-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786361401922362-8267-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786361401922362-8267-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786361401922362-8267-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.18.193:0
--local_ip_for_outbound_sockets=127.8.18.193
--webserver_interface=127.8.18.193
--webserver_port=0
--tserver_master_addrs=127.8.18.254:38619
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.193.keytab
--principal=kudu/127.8.18.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.18.212:46507
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786361401922362-8267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786361401922362-8267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786361401922362-8267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:35:44.017084 28206 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:35:44.017511 28206 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:35:44.017630 28206 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:35:44.025287 28206 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:35:44.025513 28206 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:35:44.025661 28206 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.18.193
I20260810 11:35:44.036126 28206 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.18.212:46507
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786361401922362-8267-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786361401922362-8267-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.193.keytab
--principal=kudu/127.8.18.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.18.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786361401922362-8267-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.18.193
--webserver_port=0
--tserver_master_addrs=127.8.18.254:38619
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.18.193
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786361401922362-8267-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:35:44.037616 28206 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:35:44.039165 28206 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:35:44.053946 28214 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:35:44.053903 28212 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:35:44.053994 28211 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:35:44.056061 28206 server_base.cc:1034] running on GCE node
I20260810 11:35:44.056890 28206 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:35:44.058136 28206 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:35:44.059420 28206 hybrid_clock.cc:648] HybridClock initialized: now 1786361744059347 us; error 70 us; skew 500 ppm
I20260810 11:35:44.059775 28206 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:35:44 dist-test-slave-srgv krb5kdc[28096](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361744, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:35:44.063612 28206 init.cc:377] Logged in from keytab as kudu/127.8.18.193@KRBTEST.COM (short username kudu)
I20260810 11:35:44.065594 28206 webserver.cc:460] Webserver started at http://127.8.18.193:46421/ using document root <none> and password file <none>
I20260810 11:35:44.066401 28206 fs_manager.cc:359] Metadata directory not provided
I20260810 11:35:44.066555 28206 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:35:44.066936 28206 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:35:44.070375 28206 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786361401922362-8267-0/minicluster-data/ts-0/data/instance:
uuid: "ec5b0aa8d3ee4166b9cd4c972fd9ff6e"
format_stamp: "Formatted at 2026-08-10 11:35:44 on dist-test-slave-srgv"
server_key: "b03064ce43ea062784338f61328bf64d"
server_key_iv: "fa8963c054d2b420c305c6a1da53082b"
server_key_version: "encryptionkey@0"
I20260810 11:35:44.071345 28206 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786361401922362-8267-0/minicluster-data/ts-0/wal/instance:
uuid: "ec5b0aa8d3ee4166b9cd4c972fd9ff6e"
format_stamp: "Formatted at 2026-08-10 11:35:44 on dist-test-slave-srgv"
server_key: "b03064ce43ea062784338f61328bf64d"
server_key_iv: "fa8963c054d2b420c305c6a1da53082b"
server_key_version: "encryptionkey@0"
I20260810 11:35:44.077661 28206 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260810 11:35:44.081820 28222 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:35:44.083515 28206 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 11:35:44.083734 28206 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786361401922362-8267-0/minicluster-data/ts-0/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786361401922362-8267-0/minicluster-data/ts-0/wal
uuid: "ec5b0aa8d3ee4166b9cd4c972fd9ff6e"
format_stamp: "Formatted at 2026-08-10 11:35:44 on dist-test-slave-srgv"
server_key: "b03064ce43ea062784338f61328bf64d"
server_key_iv: "fa8963c054d2b420c305c6a1da53082b"
server_key_version: "encryptionkey@0"
I20260810 11:35:44.083985 28206 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786361401922362-8267-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786361401922362-8267-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786361401922362-8267-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:35:44.109331 28206 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:35:44.114310 28206 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:35:44.114705 28206 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:35:44.115855 28206 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:35:44.117475 28206 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:35:44.117574 28206 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20260810 11:35:44.117729 28206 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:35:44.117821 28206 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:35:44.155498 28206 rpc_server.cc:307] RPC server started. Bound to: 127.8.18.193:46751
I20260810 11:35:44.155551 28336 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.18.193:46751 every 8 connection(s)
I20260810 11:35:44.157042 28206 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786361401922362-8267-0/minicluster-data/ts-0/data/info.pb
I20260810 11:35:44.157593 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 28206
I20260810 11:35:44.157804 8267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786361401922362-8267-0/minicluster-data/ts-0/wal/instance
I20260810 11:35:44.158381 8267 external_mini_cluster.cc:1428] Setting key 9a1a4ee469c02c0dae19a54b18a1dc67
Aug 10 11:35:44 dist-test-slave-srgv krb5kdc[28096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361744, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.193@KRBTEST.COM for kudu/127.8.18.254@KRBTEST.COM
I20260810 11:35:44.185284 28339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:44.160376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.193:34685 (local address 127.8.18.254:38619)
0810 11:35:44.161106 (+ 730us) server_negotiation.cc:206] Beginning negotiation
0810 11:35:44.161113 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:35:44.162239 (+ 1126us) server_negotiation.cc:407] Connection header received
0810 11:35:44.165489 (+ 3250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:35:44.165500 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:35:44.165594 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:35:44.165705 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:35:44.168180 (+ 2475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:44.168985 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:44.171814 (+ 2829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:44.172308 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:44.172435 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:35:44.176912 (+ 4477us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:35:44.176922 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:35:44.176930 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:35:44.176987 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:35:44.181110 (+ 4123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:35:44.182190 (+ 1080us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:35:44.182200 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:35:44.182206 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:35:44.182392 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:35:44.182997 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:35:44.183005 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:35:44.183011 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:35:44.183433 (+ 422us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:35:44.183629 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0810 11:35:44.184471 (+ 842us) server_negotiation.cc:299] Negotiation successful
0810 11:35:44.184787 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":602,"thread_start_us":482,"threads_started":1}
I20260810 11:35:44.186041 28340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:44.161070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:38619 (local address 127.8.18.193:34685)
0810 11:35:44.162032 (+ 962us) negotiation.cc:107] Waiting for socket to connect
0810 11:35:44.162087 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0810 11:35:44.165030 (+ 2943us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:35:44.166028 (+ 998us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:35:44.166044 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:35:44.166922 (+ 878us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:35:44.167857 (+ 935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:35:44.167889 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:35:44.169215 (+ 1326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:35:44.169223 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:35:44.171580 (+ 2357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:35:44.171604 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:35:44.172621 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:35:44.172631 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:35:44.173008 (+ 377us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:35:44.173925 (+ 917us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:35:44.173962 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:35:44.176594 (+ 2632us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:35:44.181364 (+ 4770us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:35:44.181373 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:35:44.181390 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:35:44.182002 (+ 612us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:35:44.182528 (+ 526us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:35:44.182531 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:35:44.182533 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:35:44.182870 (+ 337us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:35:44.183603 (+ 733us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:35:44.183610 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:35:44.184001 (+ 391us) client_negotiation.cc:769] Sending connection context
0810 11:35:44.184270 (+ 269us) client_negotiation.cc:240] Negotiation successful
0810 11:35:44.184759 (+ 489us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":641,"thread_start_us":392,"threads_started":1}
I20260810 11:35:44.188030 28337 heartbeater.cc:347] Connected to a master server at 127.8.18.254:38619
I20260810 11:35:44.188444 28337 heartbeater.cc:464] Registering TS with master...
I20260810 11:35:44.189364 28337 heartbeater.cc:511] Master 127.8.18.254:38619 requested a full tablet report, sending...
I20260810 11:35:44.191967 28141 ts_manager.cc:194] Registered new tserver with Master: ec5b0aa8d3ee4166b9cd4c972fd9ff6e (127.8.18.193:46751)
I20260810 11:35:44.193656 28141 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.18.193@KRBTEST.COM'} at 127.8.18.193:34685
WARNING: no policy specified for kudu/127.8.18.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.18.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.18.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.194.keytab.
Entry for principal kudu/127.8.18.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.194.keytab.
I20260810 11:35:44.237566 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786361401922362-8267-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786361401922362-8267-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786361401922362-8267-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786361401922362-8267-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.18.194:0
--local_ip_for_outbound_sockets=127.8.18.194
--webserver_interface=127.8.18.194
--webserver_port=0
--tserver_master_addrs=127.8.18.254:38619
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.194.keytab
--principal=kudu/127.8.18.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.18.212:46507
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786361401922362-8267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786361401922362-8267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786361401922362-8267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:35:44.624416 28343 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:35:44.624825 28343 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:35:44.625015 28343 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:35:44.633289 28343 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:35:44.633533 28343 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:35:44.633740 28343 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.18.194
I20260810 11:35:44.644513 28343 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.18.212:46507
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786361401922362-8267-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786361401922362-8267-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.194.keytab
--principal=kudu/127.8.18.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.18.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786361401922362-8267-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.18.194
--webserver_port=0
--tserver_master_addrs=127.8.18.254:38619
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.18.194
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786361401922362-8267-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:35:44.646104 28343 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:35:44.647576 28343 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:35:44.662711 28348 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:35:44.662606 28350 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:35:44.662678 28352 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:35:44.663604 28343 server_base.cc:1034] running on GCE node
I20260810 11:35:44.664376 28343 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:35:44.665513 28343 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:35:44.666865 28343 hybrid_clock.cc:648] HybridClock initialized: now 1786361744666828 us; error 40 us; skew 500 ppm
I20260810 11:35:44.667317 28343 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:35:44 dist-test-slave-srgv krb5kdc[28096](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361744, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:35:44.671219 28343 init.cc:377] Logged in from keytab as kudu/127.8.18.194@KRBTEST.COM (short username kudu)
I20260810 11:35:44.673069 28343 webserver.cc:460] Webserver started at http://127.8.18.194:41839/ using document root <none> and password file <none>
I20260810 11:35:44.673918 28343 fs_manager.cc:359] Metadata directory not provided
I20260810 11:35:44.674044 28343 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:35:44.674424 28343 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:35:44.677874 28343 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786361401922362-8267-0/minicluster-data/ts-1/data/instance:
uuid: "bde2bb895827498196f578169a5faf79"
format_stamp: "Formatted at 2026-08-10 11:35:44 on dist-test-slave-srgv"
server_key: "8672de9c830c3c883d39a89cab45d9d1"
server_key_iv: "f07331a4ed1341ba6303f2139bdb8bc5"
server_key_version: "encryptionkey@0"
I20260810 11:35:44.678911 28343 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786361401922362-8267-0/minicluster-data/ts-1/wal/instance:
uuid: "bde2bb895827498196f578169a5faf79"
format_stamp: "Formatted at 2026-08-10 11:35:44 on dist-test-slave-srgv"
server_key: "8672de9c830c3c883d39a89cab45d9d1"
server_key_iv: "f07331a4ed1341ba6303f2139bdb8bc5"
server_key_version: "encryptionkey@0"
I20260810 11:35:44.685066 28343 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260810 11:35:44.689849 28360 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:35:44.691563 28343 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260810 11:35:44.691813 28343 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786361401922362-8267-0/minicluster-data/ts-1/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786361401922362-8267-0/minicluster-data/ts-1/wal
uuid: "bde2bb895827498196f578169a5faf79"
format_stamp: "Formatted at 2026-08-10 11:35:44 on dist-test-slave-srgv"
server_key: "8672de9c830c3c883d39a89cab45d9d1"
server_key_iv: "f07331a4ed1341ba6303f2139bdb8bc5"
server_key_version: "encryptionkey@0"
I20260810 11:35:44.692072 28343 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786361401922362-8267-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786361401922362-8267-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786361401922362-8267-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:35:44.720754 28343 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:35:44.725963 28343 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:35:44.726300 28343 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:35:44.727387 28343 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:35:44.728955 28343 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:35:44.729068 28343 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:35:44.729229 28343 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:35:44.729305 28343 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:35:44.765538 28343 rpc_server.cc:307] RPC server started. Bound to: 127.8.18.194:43825
I20260810 11:35:44.765578 28475 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.18.194:43825 every 8 connection(s)
I20260810 11:35:44.767001 28343 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786361401922362-8267-0/minicluster-data/ts-1/data/info.pb
I20260810 11:35:44.773013 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 28343
I20260810 11:35:44.773236 8267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786361401922362-8267-0/minicluster-data/ts-1/wal/instance
I20260810 11:35:44.773778 8267 external_mini_cluster.cc:1428] Setting key ac58f4b6a92616a2171382b6816ff3fb
Aug 10 11:35:44 dist-test-slave-srgv krb5kdc[28096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361744, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.194@KRBTEST.COM for kudu/127.8.18.254@KRBTEST.COM
I20260810 11:35:44.792775 28478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:44.770227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.194:34135 (local address 127.8.18.254:38619)
0810 11:35:44.770734 (+ 507us) server_negotiation.cc:206] Beginning negotiation
0810 11:35:44.770739 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:35:44.771859 (+ 1120us) server_negotiation.cc:407] Connection header received
0810 11:35:44.774117 (+ 2258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:35:44.774123 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:35:44.774241 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:35:44.774370 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:35:44.776267 (+ 1897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:44.777033 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:44.778998 (+ 1965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:44.779558 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:44.779724 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:35:44.784710 (+ 4986us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:35:44.784717 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:35:44.784720 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:35:44.784763 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:35:44.788217 (+ 3454us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:35:44.789455 (+ 1238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:35:44.789465 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:35:44.789469 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:35:44.789780 (+ 311us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:35:44.790337 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:35:44.790346 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:35:44.790352 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:35:44.790824 (+ 472us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:35:44.790998 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0810 11:35:44.791924 (+ 926us) server_negotiation.cc:299] Negotiation successful
0810 11:35:44.792224 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":388,"thread_start_us":274,"threads_started":1}
I20260810 11:35:44.793711 28479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:44.770821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:38619 (local address 127.8.18.194:34135)
0810 11:35:44.771618 (+ 797us) negotiation.cc:107] Waiting for socket to connect
0810 11:35:44.771670 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0810 11:35:44.773759 (+ 2089us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:35:44.774696 (+ 937us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:35:44.774710 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:35:44.775347 (+ 637us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:35:44.776072 (+ 725us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:35:44.776101 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:35:44.777219 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:35:44.777224 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:35:44.778803 (+ 1579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:35:44.778829 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:35:44.779847 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:35:44.779855 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:35:44.780132 (+ 277us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:35:44.781003 (+ 871us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:35:44.781030 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:35:44.784339 (+ 3309us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:35:44.788514 (+ 4175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:35:44.788525 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:35:44.788541 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:35:44.789253 (+ 712us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:35:44.789956 (+ 703us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:35:44.789960 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:35:44.789962 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:35:44.790197 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:35:44.790989 (+ 792us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:35:44.790996 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:35:44.791402 (+ 406us) client_negotiation.cc:769] Sending connection context
0810 11:35:44.791772 (+ 370us) client_negotiation.cc:240] Negotiation successful
0810 11:35:44.792320 (+ 548us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":553,"thread_start_us":317,"threads_started":1}
I20260810 11:35:44.795437 28476 heartbeater.cc:347] Connected to a master server at 127.8.18.254:38619
I20260810 11:35:44.795859 28476 heartbeater.cc:464] Registering TS with master...
I20260810 11:35:44.796866 28476 heartbeater.cc:511] Master 127.8.18.254:38619 requested a full tablet report, sending...
I20260810 11:35:44.798957 28141 ts_manager.cc:194] Registered new tserver with Master: bde2bb895827498196f578169a5faf79 (127.8.18.194:43825)
I20260810 11:35:44.799902 28141 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.18.194@KRBTEST.COM'} at 127.8.18.194:34135
WARNING: no policy specified for kudu/127.8.18.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.18.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.18.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.195.keytab.
Entry for principal kudu/127.8.18.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.195.keytab.
I20260810 11:35:44.852649 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786361401922362-8267-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786361401922362-8267-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786361401922362-8267-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786361401922362-8267-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.18.195:0
--local_ip_for_outbound_sockets=127.8.18.195
--webserver_interface=127.8.18.195
--webserver_port=0
--tserver_master_addrs=127.8.18.254:38619
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.195.keytab
--principal=kudu/127.8.18.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.18.212:46507
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786361401922362-8267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786361401922362-8267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786361401922362-8267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 11:35:45.197611 28337 heartbeater.cc:503] Master 127.8.18.254:38619 was elected leader, sending a full tablet report...
W20260810 11:35:45.221278 28482 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:35:45.221733 28482 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:35:45.221912 28482 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:35:45.229743 28482 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:35:45.229921 28482 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:35:45.230113 28482 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.18.195
I20260810 11:35:45.240643 28482 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.18.212:46507
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786361401922362-8267-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786361401922362-8267-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.195.keytab
--principal=kudu/127.8.18.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.18.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786361401922362-8267-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.18.195
--webserver_port=0
--tserver_master_addrs=127.8.18.254:38619
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.18.195
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786361401922362-8267-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:35:45.242210 28482 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:35:45.243645 28482 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:35:45.258587 28487 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:35:45.258560 28488 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:35:45.258659 28490 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:35:45.260038 28482 server_base.cc:1034] running on GCE node
I20260810 11:35:45.260787 28482 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:35:45.262004 28482 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:35:45.263280 28482 hybrid_clock.cc:648] HybridClock initialized: now 1786361745263227 us; error 58 us; skew 500 ppm
I20260810 11:35:45.263659 28482 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:35:45 dist-test-slave-srgv krb5kdc[28096](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361745, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:35:45.267369 28482 init.cc:377] Logged in from keytab as kudu/127.8.18.195@KRBTEST.COM (short username kudu)
I20260810 11:35:45.269119 28482 webserver.cc:460] Webserver started at http://127.8.18.195:37049/ using document root <none> and password file <none>
I20260810 11:35:45.269966 28482 fs_manager.cc:359] Metadata directory not provided
I20260810 11:35:45.270129 28482 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:35:45.270473 28482 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:35:45.273749 28482 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786361401922362-8267-0/minicluster-data/ts-2/data/instance:
uuid: "3b3f243d27e14223b065d36e39240504"
format_stamp: "Formatted at 2026-08-10 11:35:45 on dist-test-slave-srgv"
server_key: "6334293c84da822333e1a0484bdfc42c"
server_key_iv: "1304da5c1e2c59900bb9f6374606cdd0"
server_key_version: "encryptionkey@0"
I20260810 11:35:45.274704 28482 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786361401922362-8267-0/minicluster-data/ts-2/wal/instance:
uuid: "3b3f243d27e14223b065d36e39240504"
format_stamp: "Formatted at 2026-08-10 11:35:45 on dist-test-slave-srgv"
server_key: "6334293c84da822333e1a0484bdfc42c"
server_key_iv: "1304da5c1e2c59900bb9f6374606cdd0"
server_key_version: "encryptionkey@0"
I20260810 11:35:45.280616 28482 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260810 11:35:45.284828 28499 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:35:45.286556 28482 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 11:35:45.286770 28482 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786361401922362-8267-0/minicluster-data/ts-2/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786361401922362-8267-0/minicluster-data/ts-2/wal
uuid: "3b3f243d27e14223b065d36e39240504"
format_stamp: "Formatted at 2026-08-10 11:35:45 on dist-test-slave-srgv"
server_key: "6334293c84da822333e1a0484bdfc42c"
server_key_iv: "1304da5c1e2c59900bb9f6374606cdd0"
server_key_version: "encryptionkey@0"
I20260810 11:35:45.287029 28482 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786361401922362-8267-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786361401922362-8267-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786361401922362-8267-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:35:45.308123 28482 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:35:45.312979 28482 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:35:45.313335 28482 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:35:45.314492 28482 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:35:45.316155 28482 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:35:45.316269 28482 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:35:45.316439 28482 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:35:45.316519 28482 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:35:45.354048 28482 rpc_server.cc:307] RPC server started. Bound to: 127.8.18.195:35693
I20260810 11:35:45.354116 28613 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.18.195:35693 every 8 connection(s)
I20260810 11:35:45.355477 28482 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786361401922362-8267-0/minicluster-data/ts-2/data/info.pb
I20260810 11:35:45.356236 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 28482
I20260810 11:35:45.356438 8267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786361401922362-8267-0/minicluster-data/ts-2/wal/instance
I20260810 11:35:45.357060 8267 external_mini_cluster.cc:1428] Setting key 491e0316aef0a80919cb8a6261f5ee06
Aug 10 11:35:45 dist-test-slave-srgv krb5kdc[28096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361745, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.195@KRBTEST.COM for kudu/127.8.18.254@KRBTEST.COM
I20260810 11:35:45.379519 28616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:45.358539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.195:48693 (local address 127.8.18.254:38619)
0810 11:35:45.359561 (+ 1022us) server_negotiation.cc:206] Beginning negotiation
0810 11:35:45.359568 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:35:45.360286 (+ 718us) server_negotiation.cc:407] Connection header received
0810 11:35:45.362455 (+ 2169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:35:45.362459 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:35:45.362553 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:35:45.362656 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:35:45.364313 (+ 1657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:45.365114 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:45.366872 (+ 1758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:45.367330 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:45.367437 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:35:45.372042 (+ 4605us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:35:45.372048 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:35:45.372052 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:35:45.372098 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:35:45.375785 (+ 3687us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:35:45.376694 (+ 909us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:35:45.376700 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:35:45.376703 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:35:45.376860 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:35:45.377341 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:35:45.377344 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:35:45.377346 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:35:45.377746 (+ 400us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:35:45.377888 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0810 11:35:45.378843 (+ 955us) server_negotiation.cc:299] Negotiation successful
0810 11:35:45.379062 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":856,"thread_start_us":718,"threads_started":1}
I20260810 11:35:45.380553 28617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:45.359100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:38619 (local address 127.8.18.195:48693)
0810 11:35:45.360088 (+ 988us) negotiation.cc:107] Waiting for socket to connect
0810 11:35:45.360153 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0810 11:35:45.362155 (+ 2002us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:35:45.362894 (+ 739us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:35:45.362903 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:35:45.363502 (+ 599us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:35:45.364132 (+ 630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:35:45.364151 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:35:45.365306 (+ 1155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:35:45.365310 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:35:45.366711 (+ 1401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:35:45.366725 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:35:45.367567 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:35:45.367572 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:35:45.367799 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:35:45.368594 (+ 795us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:35:45.368627 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:35:45.371775 (+ 3148us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:35:45.375969 (+ 4194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:35:45.375978 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:35:45.375993 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:35:45.376533 (+ 540us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:35:45.376978 (+ 445us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:35:45.376981 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:35:45.376983 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:35:45.377224 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:35:45.377882 (+ 658us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:35:45.377889 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:35:45.378384 (+ 495us) client_negotiation.cc:769] Sending connection context
0810 11:35:45.378647 (+ 263us) client_negotiation.cc:240] Negotiation successful
0810 11:35:45.379113 (+ 466us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":644,"thread_start_us":399,"threads_started":1}
I20260810 11:35:45.382393 28614 heartbeater.cc:347] Connected to a master server at 127.8.18.254:38619
I20260810 11:35:45.382753 28614 heartbeater.cc:464] Registering TS with master...
I20260810 11:35:45.383596 28614 heartbeater.cc:511] Master 127.8.18.254:38619 requested a full tablet report, sending...
I20260810 11:35:45.385569 28140 ts_manager.cc:194] Registered new tserver with Master: 3b3f243d27e14223b065d36e39240504 (127.8.18.195:35693)
I20260810 11:35:45.386861 28140 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.18.195@KRBTEST.COM'} at 127.8.18.195:48693
I20260810 11:35:45.387128 8267 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 11:35:45.408778 28616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:45.392898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39270 (local address 127.8.18.254:38619)
0810 11:35:45.393216 (+ 318us) server_negotiation.cc:206] Beginning negotiation
0810 11:35:45.393227 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 11:35:45.393697 (+ 470us) server_negotiation.cc:407] Connection header received
0810 11:35:45.393865 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:35:45.393870 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:35:45.393949 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:35:45.394041 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:35:45.395808 (+ 1767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:45.396689 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:45.398529 (+ 1840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:45.398965 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:45.399076 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:35:45.402102 (+ 3026us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:35:45.402109 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:35:45.402112 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:35:45.402154 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:35:45.405764 (+ 3610us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:35:45.406654 (+ 890us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:35:45.406658 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:35:45.406661 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:35:45.406803 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:35:45.407257 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:35:45.407260 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:35:45.407262 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:35:45.407622 (+ 360us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:35:45.407832 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0810 11:35:45.408222 (+ 390us) server_negotiation.cc:299] Negotiation successful
0810 11:35:45.408450 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110}
I20260810 11:35:45.413852 28140 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39270:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 11:35:45.421957 28140 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 11:35:45.460878 28628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:45.437220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.193:46751 (local address 127.0.0.1:40436)
0810 11:35:45.447142 (+ 9922us) negotiation.cc:107] Waiting for socket to connect
0810 11:35:45.447199 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0810 11:35:45.447437 (+ 238us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:35:45.449548 (+ 2111us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:35:45.449559 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:35:45.449597 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:35:45.450161 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:35:45.450183 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:35:45.453286 (+ 3103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:35:45.453297 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:35:45.457112 (+ 3815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:35:45.457127 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:35:45.459590 (+ 2463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:35:45.459597 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:35:45.459870 (+ 273us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:35:45.459969 (+ 99us) client_negotiation.cc:769] Sending connection context
0810 11:35:45.460099 (+ 130us) client_negotiation.cc:240] Negotiation successful
0810 11:35:45.460318 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9584,"mutex_wait_us":4,"spinlock_wait_cycles":48512,"thread_start_us":9439,"threads_started":1}
I20260810 11:35:45.461767 28629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:45.437366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.194:43825 (local address 127.0.0.1:44988)
0810 11:35:45.449433 (+ 12067us) negotiation.cc:107] Waiting for socket to connect
0810 11:35:45.449454 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 11:35:45.449714 (+ 260us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:35:45.452590 (+ 2876us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:35:45.452598 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:35:45.452641 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:35:45.453266 (+ 625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:35:45.453283 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:35:45.455472 (+ 2189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:35:45.455481 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:35:45.458461 (+ 2980us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:35:45.458476 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:35:45.460811 (+ 2335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:35:45.460821 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:35:45.461030 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:35:45.461062 (+ 32us) client_negotiation.cc:769] Sending connection context
0810 11:35:45.461150 (+ 88us) client_negotiation.cc:240] Negotiation successful
0810 11:35:45.461303 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11904,"thread_start_us":11721,"threads_started":1}
I20260810 11:35:45.461782 28631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:45.438429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40436 (local address 127.8.18.193:46751)
0810 11:35:45.446639 (+ 8210us) server_negotiation.cc:206] Beginning negotiation
0810 11:35:45.446653 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 11:35:45.447726 (+ 1073us) server_negotiation.cc:407] Connection header received
0810 11:35:45.448435 (+ 709us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:35:45.448450 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:35:45.448903 (+ 453us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:35:45.449165 (+ 262us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:35:45.450457 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:45.452996 (+ 2539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:45.457392 (+ 4396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:45.459355 (+ 1963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:45.459539 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:35:45.459815 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0810 11:35:45.461201 (+ 1386us) server_negotiation.cc:299] Negotiation successful
0810 11:35:45.461433 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7982,"thread_start_us":7799,"threads_started":1}
I20260810 11:35:45.462590 28630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:45.437837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44988 (local address 127.8.18.194:43825)
0810 11:35:45.445948 (+ 8111us) server_negotiation.cc:206] Beginning negotiation
0810 11:35:45.445964 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0810 11:35:45.449579 (+ 3615us) server_negotiation.cc:407] Connection header received
0810 11:35:45.450405 (+ 826us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:35:45.450418 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:35:45.450757 (+ 339us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:35:45.450973 (+ 216us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:35:45.453610 (+ 2637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:45.455208 (+ 1598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:45.458675 (+ 3467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:45.460284 (+ 1609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:45.460427 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:35:45.460618 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0810 11:35:45.462076 (+ 1458us) server_negotiation.cc:299] Negotiation successful
0810 11:35:45.462278 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7891,"thread_start_us":7762,"threads_started":1}
I20260810 11:35:45.466355 28632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:45.449952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.195:35693 (local address 127.0.0.1:44072)
0810 11:35:45.456804 (+ 6852us) negotiation.cc:107] Waiting for socket to connect
0810 11:35:45.456837 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0810 11:35:45.456981 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:35:45.458922 (+ 1941us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:35:45.458927 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:35:45.458965 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:35:45.459364 (+ 399us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:35:45.459377 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:35:45.461366 (+ 1989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:35:45.461372 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:35:45.463352 (+ 1980us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:35:45.463366 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:35:45.465476 (+ 2110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:35:45.465486 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:35:45.465667 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:35:45.465703 (+ 36us) client_negotiation.cc:769] Sending connection context
0810 11:35:45.465827 (+ 124us) client_negotiation.cc:240] Negotiation successful
0810 11:35:45.465980 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6657,"thread_start_us":7140,"threads_started":1}
I20260810 11:35:45.466463 28633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:45.450104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44072 (local address 127.8.18.195:35693)
0810 11:35:45.457953 (+ 7849us) server_negotiation.cc:206] Beginning negotiation
0810 11:35:45.457964 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 11:35:45.458007 (+ 43us) server_negotiation.cc:407] Connection header received
0810 11:35:45.458346 (+ 339us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:35:45.458356 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:35:45.458652 (+ 296us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:35:45.458917 (+ 265us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:35:45.459578 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:45.461210 (+ 1632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:45.463614 (+ 2404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:45.465163 (+ 1549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:45.465326 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:35:45.465604 (+ 278us) server_negotiation.cc:1035] Waiting for connection context
0810 11:35:45.465920 (+ 316us) server_negotiation.cc:299] Negotiation successful
0810 11:35:45.466096 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7627,"thread_start_us":7443,"threads_started":1}
I20260810 11:35:45.466506 28409 tablet_service.cc:1467] Processing CreateTablet for tablet fd284b91a2bf4a89bb3482a88bb847aa (DEFAULT_TABLE table=test-table [id=974464c9fdd04351ae8bbaf0c418541d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:35:45.466572 28272 tablet_service.cc:1467] Processing CreateTablet for tablet fd284b91a2bf4a89bb3482a88bb847aa (DEFAULT_TABLE table=test-table [id=974464c9fdd04351ae8bbaf0c418541d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:35:45.468240 28409 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fd284b91a2bf4a89bb3482a88bb847aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:35:45.468236 28272 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fd284b91a2bf4a89bb3482a88bb847aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:35:45.470956 28548 tablet_service.cc:1467] Processing CreateTablet for tablet fd284b91a2bf4a89bb3482a88bb847aa (DEFAULT_TABLE table=test-table [id=974464c9fdd04351ae8bbaf0c418541d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:35:45.472676 28548 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fd284b91a2bf4a89bb3482a88bb847aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:35:45.481235 28634 tablet_bootstrap.cc:492] T fd284b91a2bf4a89bb3482a88bb847aa P ec5b0aa8d3ee4166b9cd4c972fd9ff6e: Bootstrap starting.
I20260810 11:35:45.484786 28634 tablet_bootstrap.cc:654] T fd284b91a2bf4a89bb3482a88bb847aa P ec5b0aa8d3ee4166b9cd4c972fd9ff6e: Neither blocks nor log segments found. Creating new log.
I20260810 11:35:45.486449 28634 log.cc:824] T fd284b91a2bf4a89bb3482a88bb847aa P ec5b0aa8d3ee4166b9cd4c972fd9ff6e: Log is configured to *not* fsync() on all Append() calls
I20260810 11:35:45.486548 28635 tablet_bootstrap.cc:492] T fd284b91a2bf4a89bb3482a88bb847aa P bde2bb895827498196f578169a5faf79: Bootstrap starting.
I20260810 11:35:45.491880 28635 tablet_bootstrap.cc:654] T fd284b91a2bf4a89bb3482a88bb847aa P bde2bb895827498196f578169a5faf79: Neither blocks nor log segments found. Creating new log.
I20260810 11:35:45.493925 28635 log.cc:824] T fd284b91a2bf4a89bb3482a88bb847aa P bde2bb895827498196f578169a5faf79: Log is configured to *not* fsync() on all Append() calls
I20260810 11:35:45.494794 28637 tablet_bootstrap.cc:492] T fd284b91a2bf4a89bb3482a88bb847aa P 3b3f243d27e14223b065d36e39240504: Bootstrap starting.
I20260810 11:35:45.497810 28634 tablet_bootstrap.cc:492] T fd284b91a2bf4a89bb3482a88bb847aa P ec5b0aa8d3ee4166b9cd4c972fd9ff6e: No bootstrap required, opened a new log
I20260810 11:35:45.497861 28635 tablet_bootstrap.cc:492] T fd284b91a2bf4a89bb3482a88bb847aa P bde2bb895827498196f578169a5faf79: No bootstrap required, opened a new log
I20260810 11:35:45.498172 28635 ts_tablet_manager.cc:1397] T fd284b91a2bf4a89bb3482a88bb847aa P bde2bb895827498196f578169a5faf79: Time spent bootstrapping tablet: real 0.012s user 0.006s sys 0.005s
I20260810 11:35:45.498163 28634 ts_tablet_manager.cc:1397] T fd284b91a2bf4a89bb3482a88bb847aa P ec5b0aa8d3ee4166b9cd4c972fd9ff6e: Time spent bootstrapping tablet: real 0.017s user 0.007s sys 0.007s
I20260810 11:35:45.499191 28637 tablet_bootstrap.cc:654] T fd284b91a2bf4a89bb3482a88bb847aa P 3b3f243d27e14223b065d36e39240504: Neither blocks nor log segments found. Creating new log.
I20260810 11:35:45.500640 28637 log.cc:824] T fd284b91a2bf4a89bb3482a88bb847aa P 3b3f243d27e14223b065d36e39240504: Log is configured to *not* fsync() on all Append() calls
I20260810 11:35:45.506011 28637 tablet_bootstrap.cc:492] T fd284b91a2bf4a89bb3482a88bb847aa P 3b3f243d27e14223b065d36e39240504: No bootstrap required, opened a new log
I20260810 11:35:45.506415 28637 ts_tablet_manager.cc:1397] T fd284b91a2bf4a89bb3482a88bb847aa P 3b3f243d27e14223b065d36e39240504: Time spent bootstrapping tablet: real 0.012s user 0.003s sys 0.006s
I20260810 11:35:45.507613 28635 raft_consensus.cc:348] T fd284b91a2bf4a89bb3482a88bb847aa P bde2bb895827498196f578169a5faf79 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec5b0aa8d3ee4166b9cd4c972fd9ff6e" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 46751 } } peers { permanent_uuid: "bde2bb895827498196f578169a5faf79" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 43825 } } peers { permanent_uuid: "3b3f243d27e14223b065d36e39240504" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 35693 } }
I20260810 11:35:45.508083 28635 raft_consensus.cc:374] T fd284b91a2bf4a89bb3482a88bb847aa P bde2bb895827498196f578169a5faf79 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:35:45.508242 28635 raft_consensus.cc:729] T fd284b91a2bf4a89bb3482a88bb847aa P bde2bb895827498196f578169a5faf79 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bde2bb895827498196f578169a5faf79, State: Initialized, Role: FOLLOWER
I20260810 11:35:45.508956 28635 consensus_queue.cc:260] T fd284b91a2bf4a89bb3482a88bb847aa P bde2bb895827498196f578169a5faf79 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec5b0aa8d3ee4166b9cd4c972fd9ff6e" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 46751 } } peers { permanent_uuid: "bde2bb895827498196f578169a5faf79" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 43825 } } peers { permanent_uuid: "3b3f243d27e14223b065d36e39240504" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 35693 } }
I20260810 11:35:45.510340 28476 heartbeater.cc:503] Master 127.8.18.254:38619 was elected leader, sending a full tablet report...
I20260810 11:35:45.510229 28634 raft_consensus.cc:348] T fd284b91a2bf4a89bb3482a88bb847aa P ec5b0aa8d3ee4166b9cd4c972fd9ff6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec5b0aa8d3ee4166b9cd4c972fd9ff6e" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 46751 } } peers { permanent_uuid: "bde2bb895827498196f578169a5faf79" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 43825 } } peers { permanent_uuid: "3b3f243d27e14223b065d36e39240504" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 35693 } }
I20260810 11:35:45.510751 28634 raft_consensus.cc:374] T fd284b91a2bf4a89bb3482a88bb847aa P ec5b0aa8d3ee4166b9cd4c972fd9ff6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:35:45.510870 28634 raft_consensus.cc:729] T fd284b91a2bf4a89bb3482a88bb847aa P ec5b0aa8d3ee4166b9cd4c972fd9ff6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec5b0aa8d3ee4166b9cd4c972fd9ff6e, State: Initialized, Role: FOLLOWER
I20260810 11:35:45.511163 28635 ts_tablet_manager.cc:1428] T fd284b91a2bf4a89bb3482a88bb847aa P bde2bb895827498196f578169a5faf79: Time spent starting tablet: real 0.013s user 0.008s sys 0.004s
I20260810 11:35:45.511639 28634 consensus_queue.cc:260] T fd284b91a2bf4a89bb3482a88bb847aa P ec5b0aa8d3ee4166b9cd4c972fd9ff6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec5b0aa8d3ee4166b9cd4c972fd9ff6e" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 46751 } } peers { permanent_uuid: "bde2bb895827498196f578169a5faf79" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 43825 } } peers { permanent_uuid: "3b3f243d27e14223b065d36e39240504" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 35693 } }
I20260810 11:35:45.515127 28634 ts_tablet_manager.cc:1428] T fd284b91a2bf4a89bb3482a88bb847aa P ec5b0aa8d3ee4166b9cd4c972fd9ff6e: Time spent starting tablet: real 0.017s user 0.009s sys 0.007s
W20260810 11:35:45.519371 28477 tablet.cc:2369] T fd284b91a2bf4a89bb3482a88bb847aa P bde2bb895827498196f578169a5faf79: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 11:35:45.519554 28637 raft_consensus.cc:348] T fd284b91a2bf4a89bb3482a88bb847aa P 3b3f243d27e14223b065d36e39240504 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec5b0aa8d3ee4166b9cd4c972fd9ff6e" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 46751 } } peers { permanent_uuid: "bde2bb895827498196f578169a5faf79" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 43825 } } peers { permanent_uuid: "3b3f243d27e14223b065d36e39240504" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 35693 } }
I20260810 11:35:45.520202 28637 raft_consensus.cc:374] T fd284b91a2bf4a89bb3482a88bb847aa P 3b3f243d27e14223b065d36e39240504 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:35:45.520359 28637 raft_consensus.cc:729] T fd284b91a2bf4a89bb3482a88bb847aa P 3b3f243d27e14223b065d36e39240504 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b3f243d27e14223b065d36e39240504, State: Initialized, Role: FOLLOWER
I20260810 11:35:45.521037 28637 consensus_queue.cc:260] T fd284b91a2bf4a89bb3482a88bb847aa P 3b3f243d27e14223b065d36e39240504 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec5b0aa8d3ee4166b9cd4c972fd9ff6e" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 46751 } } peers { permanent_uuid: "bde2bb895827498196f578169a5faf79" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 43825 } } peers { permanent_uuid: "3b3f243d27e14223b065d36e39240504" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 35693 } }
I20260810 11:35:45.522197 28614 heartbeater.cc:503] Master 127.8.18.254:38619 was elected leader, sending a full tablet report...
I20260810 11:35:45.522747 28637 ts_tablet_manager.cc:1428] T fd284b91a2bf4a89bb3482a88bb847aa P 3b3f243d27e14223b065d36e39240504: Time spent starting tablet: real 0.016s user 0.012s sys 0.003s
I20260810 11:35:45.590389 28640 raft_consensus.cc:482] T fd284b91a2bf4a89bb3482a88bb847aa P bde2bb895827498196f578169a5faf79 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 11:35:45.590960 28640 raft_consensus.cc:504] T fd284b91a2bf4a89bb3482a88bb847aa P bde2bb895827498196f578169a5faf79 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec5b0aa8d3ee4166b9cd4c972fd9ff6e" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 46751 } } peers { permanent_uuid: "bde2bb895827498196f578169a5faf79" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 43825 } } peers { permanent_uuid: "3b3f243d27e14223b065d36e39240504" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 35693 } }
I20260810 11:35:45.592790 28640 leader_election.cc:288] T fd284b91a2bf4a89bb3482a88bb847aa P bde2bb895827498196f578169a5faf79 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ec5b0aa8d3ee4166b9cd4c972fd9ff6e (127.8.18.193:46751), 3b3f243d27e14223b065d36e39240504 (127.8.18.195:35693)
I20260810 11:35:45.601588 28631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:45.594057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.194:34255 (local address 127.8.18.193:46751)
0810 11:35:45.594425 (+ 368us) server_negotiation.cc:206] Beginning negotiation
0810 11:35:45.594433 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:35:45.595372 (+ 939us) server_negotiation.cc:407] Connection header received
0810 11:35:45.595572 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:35:45.595578 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:35:45.595685 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:35:45.595789 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:35:45.596538 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:45.597481 (+ 943us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:45.599524 (+ 2043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:45.600383 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:45.600600 (+ 217us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:35:45.600699 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0810 11:35:45.601121 (+ 422us) server_negotiation.cc:299] Negotiation successful
0810 11:35:45.601275 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":183}
I20260810 11:35:45.601626 28633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:45.593791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.194:44007 (local address 127.8.18.195:35693)
0810 11:35:45.594058 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0810 11:35:45.594066 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:35:45.595207 (+ 1141us) server_negotiation.cc:407] Connection header received
0810 11:35:45.595612 (+ 405us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:35:45.595628 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:35:45.595710 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:35:45.595833 (+ 123us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:35:45.596691 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:45.597710 (+ 1019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:45.599582 (+ 1872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:45.600399 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:45.600632 (+ 233us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:35:45.600714 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0810 11:35:45.601162 (+ 448us) server_negotiation.cc:299] Negotiation successful
0810 11:35:45.601307 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
I20260810 11:35:45.601891 28643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:45.593088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.195:35693 (local address 127.8.18.194:44007)
0810 11:35:45.595060 (+ 1972us) negotiation.cc:107] Waiting for socket to connect
0810 11:35:45.595084 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0810 11:35:45.595317 (+ 233us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:35:45.595867 (+ 550us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:35:45.595872 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:35:45.595897 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:35:45.596499 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:35:45.596516 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:35:45.597878 (+ 1362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:35:45.597882 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:35:45.599356 (+ 1474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:35:45.599370 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:35:45.600700 (+ 1330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:35:45.600707 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:35:45.600948 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:35:45.600973 (+ 25us) client_negotiation.cc:769] Sending connection context
0810 11:35:45.601095 (+ 122us) client_negotiation.cc:240] Negotiation successful
0810 11:35:45.601244 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1802,"mutex_wait_us":28,"thread_start_us":392,"threads_started":1}
I20260810 11:35:45.601858 28644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:45.593245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.193:46751 (local address 127.8.18.194:34255)
0810 11:35:45.595223 (+ 1978us) negotiation.cc:107] Waiting for socket to connect
0810 11:35:45.595244 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 11:35:45.595417 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:35:45.595834 (+ 417us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:35:45.595840 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:35:45.595877 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:35:45.596344 (+ 467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:35:45.596359 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:35:45.597623 (+ 1264us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:35:45.597629 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:35:45.599315 (+ 1686us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:35:45.599330 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:35:45.600663 (+ 1333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:35:45.600670 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:35:45.600918 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:35:45.600957 (+ 39us) client_negotiation.cc:769] Sending connection context
0810 11:35:45.601045 (+ 88us) client_negotiation.cc:240] Negotiation successful
0810 11:35:45.601239 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1820,"mutex_wait_us":25,"thread_start_us":1523,"threads_started":1}
I20260810 11:35:45.603171 28568 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd284b91a2bf4a89bb3482a88bb847aa" candidate_uuid: "bde2bb895827498196f578169a5faf79" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b3f243d27e14223b065d36e39240504" is_pre_election: true
I20260810 11:35:45.603233 28292 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd284b91a2bf4a89bb3482a88bb847aa" candidate_uuid: "bde2bb895827498196f578169a5faf79" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec5b0aa8d3ee4166b9cd4c972fd9ff6e" is_pre_election: true
I20260810 11:35:45.603796 28568 raft_consensus.cc:2454] T fd284b91a2bf4a89bb3482a88bb847aa P 3b3f243d27e14223b065d36e39240504 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bde2bb895827498196f578169a5faf79 in term 0.
I20260810 11:35:45.603873 28292 raft_consensus.cc:2454] T fd284b91a2bf4a89bb3482a88bb847aa P ec5b0aa8d3ee4166b9cd4c972fd9ff6e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bde2bb895827498196f578169a5faf79 in term 0.
I20260810 11:35:45.604715 28361 leader_election.cc:302] T fd284b91a2bf4a89bb3482a88bb847aa P bde2bb895827498196f578169a5faf79 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b3f243d27e14223b065d36e39240504, bde2bb895827498196f578169a5faf79; no voters:
I20260810 11:35:45.605160 28640 raft_consensus.cc:2781] T fd284b91a2bf4a89bb3482a88bb847aa P bde2bb895827498196f578169a5faf79 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 11:35:45.605327 28640 raft_consensus.cc:482] T fd284b91a2bf4a89bb3482a88bb847aa P bde2bb895827498196f578169a5faf79 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 11:35:45.605479 28640 raft_consensus.cc:3037] T fd284b91a2bf4a89bb3482a88bb847aa P bde2bb895827498196f578169a5faf79 [term 0 FOLLOWER]: Advancing to term 1
W20260810 11:35:45.607497 28615 tablet.cc:2369] T fd284b91a2bf4a89bb3482a88bb847aa P 3b3f243d27e14223b065d36e39240504: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 11:35:45.608316 28640 raft_consensus.cc:504] T fd284b91a2bf4a89bb3482a88bb847aa P bde2bb895827498196f578169a5faf79 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec5b0aa8d3ee4166b9cd4c972fd9ff6e" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 46751 } } peers { permanent_uuid: "bde2bb895827498196f578169a5faf79" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 43825 } } peers { permanent_uuid: "3b3f243d27e14223b065d36e39240504" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 35693 } }
I20260810 11:35:45.609166 28640 leader_election.cc:288] T fd284b91a2bf4a89bb3482a88bb847aa P bde2bb895827498196f578169a5faf79 [CANDIDATE]: Term 1 election: Requested vote from peers ec5b0aa8d3ee4166b9cd4c972fd9ff6e (127.8.18.193:46751), 3b3f243d27e14223b065d36e39240504 (127.8.18.195:35693)
I20260810 11:35:45.609862 28292 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd284b91a2bf4a89bb3482a88bb847aa" candidate_uuid: "bde2bb895827498196f578169a5faf79" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec5b0aa8d3ee4166b9cd4c972fd9ff6e"
I20260810 11:35:45.609910 28568 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd284b91a2bf4a89bb3482a88bb847aa" candidate_uuid: "bde2bb895827498196f578169a5faf79" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b3f243d27e14223b065d36e39240504"
I20260810 11:35:45.610112 28292 raft_consensus.cc:3037] T fd284b91a2bf4a89bb3482a88bb847aa P ec5b0aa8d3ee4166b9cd4c972fd9ff6e [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:35:45.610165 28568 raft_consensus.cc:3037] T fd284b91a2bf4a89bb3482a88bb847aa P 3b3f243d27e14223b065d36e39240504 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:35:45.612577 28292 raft_consensus.cc:2454] T fd284b91a2bf4a89bb3482a88bb847aa P ec5b0aa8d3ee4166b9cd4c972fd9ff6e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bde2bb895827498196f578169a5faf79 in term 1.
I20260810 11:35:45.612633 28568 raft_consensus.cc:2454] T fd284b91a2bf4a89bb3482a88bb847aa P 3b3f243d27e14223b065d36e39240504 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bde2bb895827498196f578169a5faf79 in term 1.
I20260810 11:35:45.613217 28361 leader_election.cc:302] T fd284b91a2bf4a89bb3482a88bb847aa P bde2bb895827498196f578169a5faf79 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 3b3f243d27e14223b065d36e39240504, bde2bb895827498196f578169a5faf79, ec5b0aa8d3ee4166b9cd4c972fd9ff6e; no voters:
I20260810 11:35:45.613703 28640 raft_consensus.cc:2781] T fd284b91a2bf4a89bb3482a88bb847aa P bde2bb895827498196f578169a5faf79 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:35:45.614238 28640 raft_consensus.cc:686] T fd284b91a2bf4a89bb3482a88bb847aa P bde2bb895827498196f578169a5faf79 [term 1 LEADER]: Becoming Leader. State: Replica: bde2bb895827498196f578169a5faf79, State: Running, Role: LEADER
I20260810 11:35:45.614821 28640 consensus_queue.cc:237] T fd284b91a2bf4a89bb3482a88bb847aa P bde2bb895827498196f578169a5faf79 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec5b0aa8d3ee4166b9cd4c972fd9ff6e" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 46751 } } peers { permanent_uuid: "bde2bb895827498196f578169a5faf79" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 43825 } } peers { permanent_uuid: "3b3f243d27e14223b065d36e39240504" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 35693 } }
I20260810 11:35:45.619690 28140 catalog_manager.cc:5495] T fd284b91a2bf4a89bb3482a88bb847aa P bde2bb895827498196f578169a5faf79 reported cstate change: term changed from 0 to 1, leader changed from <none> to bde2bb895827498196f578169a5faf79 (127.8.18.194). New cstate: current_term: 1 leader_uuid: "bde2bb895827498196f578169a5faf79" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec5b0aa8d3ee4166b9cd4c972fd9ff6e" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 46751 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bde2bb895827498196f578169a5faf79" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 43825 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b3f243d27e14223b065d36e39240504" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 35693 } health_report { overall_health: UNKNOWN } } }
I20260810 11:35:45.646334 28630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:45.640547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44998 (local address 127.8.18.194:43825)
0810 11:35:45.640818 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0810 11:35:45.640828 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 11:35:45.640844 (+ 16us) server_negotiation.cc:407] Connection header received
0810 11:35:45.640957 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:35:45.640961 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:35:45.641065 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:35:45.641254 (+ 189us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 11:35:45.641716 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:45.642469 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:45.643905 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:45.644367 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:45.644514 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:35:45.645048 (+ 534us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 11:35:45.645669 (+ 621us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 11:35:45.645835 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0810 11:35:45.645933 (+ 98us) server_negotiation.cc:299] Negotiation successful
0810 11:35:45.646026 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107}
I20260810 11:35:45.655702 28568 raft_consensus.cc:1261] T fd284b91a2bf4a89bb3482a88bb847aa P 3b3f243d27e14223b065d36e39240504 [term 1 FOLLOWER]: Refusing update from remote peer bde2bb895827498196f578169a5faf79: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 11:35:45.655699 28292 raft_consensus.cc:1261] T fd284b91a2bf4a89bb3482a88bb847aa P ec5b0aa8d3ee4166b9cd4c972fd9ff6e [term 1 FOLLOWER]: Refusing update from remote peer bde2bb895827498196f578169a5faf79: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 11:35:45.656886 28645 consensus_queue.cc:1035] T fd284b91a2bf4a89bb3482a88bb847aa P bde2bb895827498196f578169a5faf79 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b3f243d27e14223b065d36e39240504" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 35693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 11:35:45.657312 28640 consensus_queue.cc:1035] T fd284b91a2bf4a89bb3482a88bb847aa P bde2bb895827498196f578169a5faf79 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec5b0aa8d3ee4166b9cd4c972fd9ff6e" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 46751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260810 11:35:45.660282 28338 tablet.cc:2369] T fd284b91a2bf4a89bb3482a88bb847aa P ec5b0aa8d3ee4166b9cd4c972fd9ff6e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 11:35:45.671342 28648 mvcc.cc:205] Tried to move back new op lower bound from 7316937710194008064 to 7316937710042144768. Current Snapshot: MvccSnapshot[applied={T|T < 7316937710194008064}]
I20260810 11:35:45.687415 28140 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39270:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 11:35:45.687824 28140 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39270:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 11:35:45.692968 28140 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 342e92e798c6483a9561794ae1edb4bc: Sending DeleteTablet for 3 replicas of tablet fd284b91a2bf4a89bb3482a88bb847aa
I20260810 11:35:45.694594 28272 tablet_service.cc:1514] Processing DeleteTablet for tablet fd284b91a2bf4a89bb3482a88bb847aa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:35:45 UTC) from {username='kudu', principal='kudu/127.8.18.254@KRBTEST.COM'} at 127.0.0.1:40436
I20260810 11:35:45.695027 28548 tablet_service.cc:1514] Processing DeleteTablet for tablet fd284b91a2bf4a89bb3482a88bb847aa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:35:45 UTC) from {username='kudu', principal='kudu/127.8.18.254@KRBTEST.COM'} at 127.0.0.1:44072
I20260810 11:35:45.695024 28409 tablet_service.cc:1514] Processing DeleteTablet for tablet fd284b91a2bf4a89bb3482a88bb847aa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:35:45 UTC) from {username='kudu', principal='kudu/127.8.18.254@KRBTEST.COM'} at 127.0.0.1:44988
I20260810 11:35:45.698166 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 28206
I20260810 11:35:45.706311 28658 tablet_replica.cc:330] T fd284b91a2bf4a89bb3482a88bb847aa P ec5b0aa8d3ee4166b9cd4c972fd9ff6e: stopping tablet replica
I20260810 11:35:45.707137 28658 raft_consensus.cc:2229] T fd284b91a2bf4a89bb3482a88bb847aa P ec5b0aa8d3ee4166b9cd4c972fd9ff6e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 11:35:45.707939 28658 raft_consensus.cc:2258] T fd284b91a2bf4a89bb3482a88bb847aa P ec5b0aa8d3ee4166b9cd4c972fd9ff6e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:35:45.710729 28658 ts_tablet_manager.cc:1905] T fd284b91a2bf4a89bb3482a88bb847aa P ec5b0aa8d3ee4166b9cd4c972fd9ff6e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:35:45.718869 28658 ts_tablet_manager.cc:1918] T fd284b91a2bf4a89bb3482a88bb847aa P ec5b0aa8d3ee4166b9cd4c972fd9ff6e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 11:35:45.719213 28658 log.cc:1196] T fd284b91a2bf4a89bb3482a88bb847aa P ec5b0aa8d3ee4166b9cd4c972fd9ff6e: Deleting WAL directory at /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786361401922362-8267-0/minicluster-data/ts-0/wal/wals/fd284b91a2bf4a89bb3482a88bb847aa
I20260810 11:35:45.719954 28658 ts_tablet_manager.cc:1939] T fd284b91a2bf4a89bb3482a88bb847aa P ec5b0aa8d3ee4166b9cd4c972fd9ff6e: Deleting consensus metadata
I20260810 11:35:45.722409 28127 catalog_manager.cc:4841] TS ec5b0aa8d3ee4166b9cd4c972fd9ff6e (127.8.18.193:46751): tablet fd284b91a2bf4a89bb3482a88bb847aa (table test-table [id=974464c9fdd04351ae8bbaf0c418541d]) successfully deleted
I20260810 11:35:45.724527 28659 tablet_replica.cc:330] T fd284b91a2bf4a89bb3482a88bb847aa P 3b3f243d27e14223b065d36e39240504: stopping tablet replica
I20260810 11:35:45.725234 28659 raft_consensus.cc:2229] T fd284b91a2bf4a89bb3482a88bb847aa P 3b3f243d27e14223b065d36e39240504 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 11:35:45.725973 28659 raft_consensus.cc:2258] T fd284b91a2bf4a89bb3482a88bb847aa P 3b3f243d27e14223b065d36e39240504 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:35:45.726996 28660 tablet_replica.cc:330] T fd284b91a2bf4a89bb3482a88bb847aa P bde2bb895827498196f578169a5faf79: stopping tablet replica
I20260810 11:35:45.727500 28660 raft_consensus.cc:2229] T fd284b91a2bf4a89bb3482a88bb847aa P bde2bb895827498196f578169a5faf79 [term 1 LEADER]: Raft consensus shutting down.
I20260810 11:35:45.728319 28660 raft_consensus.cc:2258] T fd284b91a2bf4a89bb3482a88bb847aa P bde2bb895827498196f578169a5faf79 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:35:45.729544 28659 ts_tablet_manager.cc:1905] T fd284b91a2bf4a89bb3482a88bb847aa P 3b3f243d27e14223b065d36e39240504: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:35:45.730587 28660 ts_tablet_manager.cc:1905] T fd284b91a2bf4a89bb3482a88bb847aa P bde2bb895827498196f578169a5faf79: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 10 11:35:45 dist-test-slave-srgv krb5kdc[28096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361742, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.18.193@KRBTEST.COM
I20260810 11:35:45.738076 28659 ts_tablet_manager.cc:1918] T fd284b91a2bf4a89bb3482a88bb847aa P 3b3f243d27e14223b065d36e39240504: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 11:35:45.738402 28659 log.cc:1196] T fd284b91a2bf4a89bb3482a88bb847aa P 3b3f243d27e14223b065d36e39240504: Deleting WAL directory at /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786361401922362-8267-0/minicluster-data/ts-2/wal/wals/fd284b91a2bf4a89bb3482a88bb847aa
I20260810 11:35:45.738605 28660 ts_tablet_manager.cc:1918] T fd284b91a2bf4a89bb3482a88bb847aa P bde2bb895827498196f578169a5faf79: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 11:35:45.738844 28660 log.cc:1196] T fd284b91a2bf4a89bb3482a88bb847aa P bde2bb895827498196f578169a5faf79: Deleting WAL directory at /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786361401922362-8267-0/minicluster-data/ts-1/wal/wals/fd284b91a2bf4a89bb3482a88bb847aa
I20260810 11:35:45.739457 28659 ts_tablet_manager.cc:1939] T fd284b91a2bf4a89bb3482a88bb847aa P 3b3f243d27e14223b065d36e39240504: Deleting consensus metadata
I20260810 11:35:45.739464 28660 ts_tablet_manager.cc:1939] T fd284b91a2bf4a89bb3482a88bb847aa P bde2bb895827498196f578169a5faf79: Deleting consensus metadata
I20260810 11:35:45.741614 28125 catalog_manager.cc:4841] TS bde2bb895827498196f578169a5faf79 (127.8.18.194:43825): tablet fd284b91a2bf4a89bb3482a88bb847aa (table test-table [id=974464c9fdd04351ae8bbaf0c418541d]) successfully deleted
I20260810 11:35:45.741940 28125 catalog_manager.cc:4841] TS 3b3f243d27e14223b065d36e39240504 (127.8.18.195:35693): tablet fd284b91a2bf4a89bb3482a88bb847aa (table test-table [id=974464c9fdd04351ae8bbaf0c418541d]) successfully deleted
I20260810 11:35:45.745756 28631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:45.700520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40452 (local address 127.8.18.193:46751)
0810 11:35:45.700801 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0810 11:35:45.700811 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 11:35:45.722998 (+ 22187us) server_negotiation.cc:407] Connection header received
0810 11:35:45.723383 (+ 385us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:35:45.723392 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:35:45.723508 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:35:45.723621 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:35:45.726413 (+ 2792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:45.727402 (+ 989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:45.730558 (+ 3156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:45.731234 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:45.731352 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:35:45.737724 (+ 6372us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:35:45.737750 (+ 26us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:35:45.737774 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:35:45.737833 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:35:45.742419 (+ 4586us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:35:45.743357 (+ 938us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:35:45.743365 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:35:45.743368 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:35:45.743519 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:35:45.743978 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:35:45.743985 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:35:45.743993 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:35:45.744405 (+ 412us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:35:45.744757 (+ 352us) server_negotiation.cc:1035] Waiting for connection context
0810 11:35:45.745096 (+ 339us) server_negotiation.cc:299] Negotiation successful
0810 11:35:45.745350 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260810 11:35:45.746547 28332 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:35:45.861318 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 28206
I20260810 11:35:45.883308 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 28343
Aug 10 11:35:45 dist-test-slave-srgv krb5kdc[28096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361742, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.18.194@KRBTEST.COM
I20260810 11:35:45.903759 28630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:45.884378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45006 (local address 127.8.18.194:43825)
0810 11:35:45.884619 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0810 11:35:45.884626 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:35:45.884649 (+ 23us) server_negotiation.cc:407] Connection header received
0810 11:35:45.884725 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:35:45.884730 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:35:45.884831 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:35:45.884948 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:35:45.886665 (+ 1717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:45.888021 (+ 1356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:45.889573 (+ 1552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:45.890345 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:45.890479 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:35:45.895631 (+ 5152us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:35:45.895650 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:35:45.895693 (+ 43us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:35:45.895748 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:35:45.900247 (+ 4499us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:35:45.901154 (+ 907us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:35:45.901166 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:35:45.901172 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:35:45.901389 (+ 217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:35:45.901956 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:35:45.901962 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:35:45.901965 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:35:45.902504 (+ 539us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:35:45.902890 (+ 386us) server_negotiation.cc:1035] Waiting for connection context
0810 11:35:45.903074 (+ 184us) server_negotiation.cc:299] Negotiation successful
0810 11:35:45.903338 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260810 11:35:45.904767 28471 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:35:46.033598 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 28343
I20260810 11:35:46.056741 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 28482
Aug 10 11:35:46 dist-test-slave-srgv krb5kdc[28096](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361742, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.18.195@KRBTEST.COM
I20260810 11:35:46.083343 28633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:46.057733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44088 (local address 127.8.18.195:35693)
0810 11:35:46.057975 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0810 11:35:46.057983 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:35:46.058010 (+ 27us) server_negotiation.cc:407] Connection header received
0810 11:35:46.058254 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:35:46.058268 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:35:46.058344 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:35:46.058486 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:35:46.060196 (+ 1710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:46.061020 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:46.062695 (+ 1675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:46.063164 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:46.063325 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:35:46.076034 (+ 12709us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:35:46.076096 (+ 62us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:35:46.076114 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:35:46.076167 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:35:46.079982 (+ 3815us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:35:46.080918 (+ 936us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:35:46.080927 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:35:46.080929 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:35:46.081095 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:35:46.081614 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:35:46.081619 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:35:46.081621 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:35:46.082023 (+ 402us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:35:46.082371 (+ 348us) server_negotiation.cc:1035] Waiting for connection context
0810 11:35:46.082700 (+ 329us) server_negotiation.cc:299] Negotiation successful
0810 11:35:46.082952 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260810 11:35:46.084048 28609 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:35:46.181310 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 28482
I20260810 11:35:46.201682 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 28109
I20260810 11:35:46.202904 28171 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:35:46.321103 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 28109
2026-08-10T11:35:46Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (4871 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-10T11:35:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T11:35:46Z Disabled control of system clock
I20260810 11:35:46.419600 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786361401922362-8267-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786361401922362-8267-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786361401922362-8267-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.18.254:42435
--webserver_interface=127.8.18.254
--webserver_port=0
--builtin_ntp_servers=127.8.18.212:35517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.18.254:42435
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 11:35:46.791157 28670 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:35:46.791486 28670 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:35:46.791572 28670 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:35:46.799088 28670 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 11:35:46.799232 28670 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:35:46.799273 28670 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:35:46.799324 28670 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 11:35:46.799368 28670 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 11:35:46.809111 28670 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.18.212:35517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786361401922362-8267-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786361401922362-8267-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.18.254:42435
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.18.254:42435
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.18.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786361401922362-8267-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:35:46.810585 28670 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:35:46.812633 28670 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:35:46.826644 28675 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:35:46.826784 28676 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:35:46.826776 28678 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:35:46.828111 28670 server_base.cc:1034] running on GCE node
I20260810 11:35:46.829188 28670 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:35:46.830950 28670 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:35:46.832238 28670 hybrid_clock.cc:648] HybridClock initialized: now 1786361746832192 us; error 80 us; skew 500 ppm
I20260810 11:35:46.832650 28670 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 11:35:46.835513 28670 webserver.cc:460] Webserver started at http://127.8.18.254:36965/ using document root <none> and password file <none>
I20260810 11:35:46.836340 28670 fs_manager.cc:359] Metadata directory not provided
I20260810 11:35:46.836493 28670 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:35:46.836853 28670 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:35:46.840161 28670 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786361401922362-8267-0/minicluster-data/master-0/data/instance:
uuid: "ea18459c3204475c9a59f534dc620eb8"
format_stamp: "Formatted at 2026-08-10 11:35:46 on dist-test-slave-srgv"
server_key: "6d7e2c6797f85403250ce9551f0f3190"
server_key_iv: "e8fac38fa477dfa3cb4cc4979018942b"
server_key_version: "encryptionkey@0"
I20260810 11:35:46.841127 28670 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786361401922362-8267-0/minicluster-data/master-0/wal/instance:
uuid: "ea18459c3204475c9a59f534dc620eb8"
format_stamp: "Formatted at 2026-08-10 11:35:46 on dist-test-slave-srgv"
server_key: "6d7e2c6797f85403250ce9551f0f3190"
server_key_iv: "e8fac38fa477dfa3cb4cc4979018942b"
server_key_version: "encryptionkey@0"
I20260810 11:35:46.847159 28670 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.002s
I20260810 11:35:46.851102 28684 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:35:46.852566 28670 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 11:35:46.852802 28670 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786361401922362-8267-0/minicluster-data/master-0/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786361401922362-8267-0/minicluster-data/master-0/wal
uuid: "ea18459c3204475c9a59f534dc620eb8"
format_stamp: "Formatted at 2026-08-10 11:35:46 on dist-test-slave-srgv"
server_key: "6d7e2c6797f85403250ce9551f0f3190"
server_key_iv: "e8fac38fa477dfa3cb4cc4979018942b"
server_key_version: "encryptionkey@0"
I20260810 11:35:46.853077 28670 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786361401922362-8267-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786361401922362-8267-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786361401922362-8267-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:35:46.876176 28670 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:35:46.877169 28670 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:35:46.877522 28670 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:35:46.901312 28670 rpc_server.cc:307] RPC server started. Bound to: 127.8.18.254:42435
I20260810 11:35:46.901326 28737 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.18.254:42435 every 8 connection(s)
I20260810 11:35:46.902877 28670 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
I20260810 11:35:46.903673 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 28670
I20260810 11:35:46.903916 8267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786361401922362-8267-0/minicluster-data/master-0/wal/instance
I20260810 11:35:46.904600 8267 external_mini_cluster.cc:1428] Setting key 4754064dbdd27e290f26c37f35251bba
I20260810 11:35:46.908397 28739 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:35:46.920069 28742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:46.906613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60624 (local address 127.8.18.254:42435)
0810 11:35:46.907958 (+ 1345us) server_negotiation.cc:206] Beginning negotiation
0810 11:35:46.907974 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0810 11:35:46.908132 (+ 158us) server_negotiation.cc:407] Connection header received
0810 11:35:46.910283 (+ 2151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:35:46.910308 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:35:46.910810 (+ 502us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:35:46.911540 (+ 730us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:35:46.912189 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:46.913640 (+ 1451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:46.915095 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:46.915810 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:46.916020 (+ 210us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:35:46.917360 (+ 1340us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:35:46.917369 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:35:46.917385 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:35:46.918128 (+ 743us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:35:46.918228 (+ 100us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:35:46.918253 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:35:46.918407 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0810 11:35:46.918710 (+ 303us) server_negotiation.cc:299] Negotiation successful
0810 11:35:46.919061 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":774,"thread_start_us":427,"threads_started":1}
I20260810 11:35:46.920259 28739 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea18459c3204475c9a59f534dc620eb8: Bootstrap starting.
I20260810 11:35:46.924511 28739 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ea18459c3204475c9a59f534dc620eb8: Neither blocks nor log segments found. Creating new log.
I20260810 11:35:46.925750 28739 log.cc:824] T 00000000000000000000000000000000 P ea18459c3204475c9a59f534dc620eb8: Log is configured to *not* fsync() on all Append() calls
I20260810 11:35:46.929522 28739 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea18459c3204475c9a59f534dc620eb8: No bootstrap required, opened a new log
I20260810 11:35:46.938410 28739 raft_consensus.cc:348] T 00000000000000000000000000000000 P ea18459c3204475c9a59f534dc620eb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea18459c3204475c9a59f534dc620eb8" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 42435 } }
I20260810 11:35:46.939019 28739 raft_consensus.cc:374] T 00000000000000000000000000000000 P ea18459c3204475c9a59f534dc620eb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:35:46.939181 28739 raft_consensus.cc:729] T 00000000000000000000000000000000 P ea18459c3204475c9a59f534dc620eb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea18459c3204475c9a59f534dc620eb8, State: Initialized, Role: FOLLOWER
I20260810 11:35:46.939881 28739 consensus_queue.cc:260] T 00000000000000000000000000000000 P ea18459c3204475c9a59f534dc620eb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea18459c3204475c9a59f534dc620eb8" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 42435 } }
I20260810 11:35:46.940141 28739 raft_consensus.cc:388] T 00000000000000000000000000000000 P ea18459c3204475c9a59f534dc620eb8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 11:35:46.940310 28739 raft_consensus.cc:482] T 00000000000000000000000000000000 P ea18459c3204475c9a59f534dc620eb8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 11:35:46.940506 28739 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ea18459c3204475c9a59f534dc620eb8 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:35:46.942945 28739 raft_consensus.cc:504] T 00000000000000000000000000000000 P ea18459c3204475c9a59f534dc620eb8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea18459c3204475c9a59f534dc620eb8" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 42435 } }
I20260810 11:35:46.943547 28739 leader_election.cc:302] T 00000000000000000000000000000000 P ea18459c3204475c9a59f534dc620eb8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ea18459c3204475c9a59f534dc620eb8; no voters:
I20260810 11:35:46.944231 28739 leader_election.cc:288] T 00000000000000000000000000000000 P ea18459c3204475c9a59f534dc620eb8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 11:35:46.944437 28744 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ea18459c3204475c9a59f534dc620eb8 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:35:46.945362 28744 raft_consensus.cc:686] T 00000000000000000000000000000000 P ea18459c3204475c9a59f534dc620eb8 [term 1 LEADER]: Becoming Leader. State: Replica: ea18459c3204475c9a59f534dc620eb8, State: Running, Role: LEADER
I20260810 11:35:46.945966 28744 consensus_queue.cc:237] T 00000000000000000000000000000000 P ea18459c3204475c9a59f534dc620eb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea18459c3204475c9a59f534dc620eb8" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 42435 } }
I20260810 11:35:46.946552 28739 sys_catalog.cc:564] T 00000000000000000000000000000000 P ea18459c3204475c9a59f534dc620eb8 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 11:35:46.948951 28745 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea18459c3204475c9a59f534dc620eb8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ea18459c3204475c9a59f534dc620eb8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea18459c3204475c9a59f534dc620eb8" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 42435 } } }
I20260810 11:35:46.949368 28745 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea18459c3204475c9a59f534dc620eb8 [sys.catalog]: This master's current role is: LEADER
I20260810 11:35:46.950510 28746 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea18459c3204475c9a59f534dc620eb8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ea18459c3204475c9a59f534dc620eb8. Latest consensus state: current_term: 1 leader_uuid: "ea18459c3204475c9a59f534dc620eb8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea18459c3204475c9a59f534dc620eb8" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 42435 } } }
I20260810 11:35:46.950920 28746 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea18459c3204475c9a59f534dc620eb8 [sys.catalog]: This master's current role is: LEADER
I20260810 11:35:46.954950 28752 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 11:35:46.961114 28752 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 11:35:46.971288 28752 catalog_manager.cc:1347] Generated new cluster ID: 187ce0a02a9b48e2bc93129216a30b0e
I20260810 11:35:46.971473 28752 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 11:35:46.983343 28752 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 11:35:46.984736 28752 catalog_manager.cc:1504] Loading token signing keys...
I20260810 11:35:46.995117 28752 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ea18459c3204475c9a59f534dc620eb8: Generated new TSK 0
I20260810 11:35:46.996080 28752 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 11:35:47.011266 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786361401922362-8267-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786361401922362-8267-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786361401922362-8267-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786361401922362-8267-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.18.193:0
--local_ip_for_outbound_sockets=127.8.18.193
--webserver_interface=127.8.18.193
--webserver_port=0
--tserver_master_addrs=127.8.18.254:42435
--builtin_ntp_servers=127.8.18.212:35517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 11:35:47.396816 28763 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:35:47.397146 28763 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:35:47.397316 28763 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:35:47.405493 28763 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:35:47.405653 28763 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:35:47.405840 28763 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.18.193
I20260810 11:35:47.415897 28763 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.18.212:35517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786361401922362-8267-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786361401922362-8267-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.18.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786361401922362-8267-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.18.193
--webserver_port=0
--tserver_master_addrs=127.8.18.254:42435
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.18.193
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786361401922362-8267-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:35:47.417356 28763 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:35:47.419406 28763 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:35:47.435144 28768 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:35:47.435158 28769 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:35:47.435596 28771 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:35:47.436084 28763 server_base.cc:1034] running on GCE node
I20260810 11:35:47.436803 28763 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:35:47.438125 28763 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:35:47.439405 28763 hybrid_clock.cc:648] HybridClock initialized: now 1786361747439357 us; error 50 us; skew 500 ppm
I20260810 11:35:47.439780 28763 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 11:35:47.442745 28763 webserver.cc:460] Webserver started at http://127.8.18.193:45915/ using document root <none> and password file <none>
I20260810 11:35:47.443578 28763 fs_manager.cc:359] Metadata directory not provided
I20260810 11:35:47.443742 28763 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:35:47.444141 28763 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:35:47.447948 28763 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786361401922362-8267-0/minicluster-data/ts-0/data/instance:
uuid: "c81ea21b94ba481fa3af5c88bc92d8c5"
format_stamp: "Formatted at 2026-08-10 11:35:47 on dist-test-slave-srgv"
server_key: "84d803f7208afcf086af5a5bcc1ed9de"
server_key_iv: "9f3a721297a4cc4c53bfad708a569104"
server_key_version: "encryptionkey@0"
I20260810 11:35:47.448997 28763 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786361401922362-8267-0/minicluster-data/ts-0/wal/instance:
uuid: "c81ea21b94ba481fa3af5c88bc92d8c5"
format_stamp: "Formatted at 2026-08-10 11:35:47 on dist-test-slave-srgv"
server_key: "84d803f7208afcf086af5a5bcc1ed9de"
server_key_iv: "9f3a721297a4cc4c53bfad708a569104"
server_key_version: "encryptionkey@0"
I20260810 11:35:47.456331 28763 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.000s
I20260810 11:35:47.461900 28777 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:35:47.463624 28763 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260810 11:35:47.463905 28763 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786361401922362-8267-0/minicluster-data/ts-0/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786361401922362-8267-0/minicluster-data/ts-0/wal
uuid: "c81ea21b94ba481fa3af5c88bc92d8c5"
format_stamp: "Formatted at 2026-08-10 11:35:47 on dist-test-slave-srgv"
server_key: "84d803f7208afcf086af5a5bcc1ed9de"
server_key_iv: "9f3a721297a4cc4c53bfad708a569104"
server_key_version: "encryptionkey@0"
I20260810 11:35:47.464304 28763 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786361401922362-8267-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786361401922362-8267-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786361401922362-8267-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:35:47.486883 28763 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:35:47.487886 28763 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:35:47.488265 28763 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:35:47.489460 28763 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:35:47.491349 28763 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:35:47.491488 28763 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:35:47.491643 28763 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:35:47.491741 28763 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:35:47.530103 28763 rpc_server.cc:307] RPC server started. Bound to: 127.8.18.193:45567
I20260810 11:35:47.530208 28898 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.18.193:45567 every 8 connection(s)
I20260810 11:35:47.531563 28763 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786361401922362-8267-0/minicluster-data/ts-0/data/info.pb
I20260810 11:35:47.539491 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 28763
I20260810 11:35:47.539703 8267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786361401922362-8267-0/minicluster-data/ts-0/wal/instance
I20260810 11:35:47.540338 8267 external_mini_cluster.cc:1428] Setting key aef229dd0aa0d6daac857071e634f3f4
I20260810 11:35:47.543833 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786361401922362-8267-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786361401922362-8267-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786361401922362-8267-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786361401922362-8267-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.18.194:0
--local_ip_for_outbound_sockets=127.8.18.194
--webserver_interface=127.8.18.194
--webserver_port=0
--tserver_master_addrs=127.8.18.254:42435
--builtin_ntp_servers=127.8.18.212:35517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260810 11:35:47.547114 28901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:47.534548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.193:48379 (local address 127.8.18.254:42435)
0810 11:35:47.535049 (+ 501us) server_negotiation.cc:206] Beginning negotiation
0810 11:35:47.535056 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:35:47.536410 (+ 1354us) server_negotiation.cc:407] Connection header received
0810 11:35:47.538449 (+ 2039us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:35:47.538454 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:35:47.538545 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:35:47.538653 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:35:47.539933 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:47.541015 (+ 1082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:47.543467 (+ 2452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:47.544193 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:47.544362 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:35:47.545994 (+ 1632us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:35:47.546002 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:35:47.546010 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:35:47.546042 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:35:47.546101 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:35:47.546120 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:35:47.546233 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0810 11:35:47.546496 (+ 263us) server_negotiation.cc:299] Negotiation successful
0810 11:35:47.546634 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":384,"thread_start_us":259,"threads_started":1}
I20260810 11:35:47.548362 28902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:47.535205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:42435 (local address 127.8.18.193:48379)
0810 11:35:47.536227 (+ 1022us) negotiation.cc:107] Waiting for socket to connect
0810 11:35:47.536275 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0810 11:35:47.538146 (+ 1871us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:35:47.538884 (+ 738us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:35:47.538893 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:35:47.538986 (+ 93us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:35:47.539685 (+ 699us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:35:47.539709 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:35:47.541214 (+ 1505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:35:47.541221 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:35:47.543273 (+ 2052us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:35:47.543290 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:35:47.544482 (+ 1192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:35:47.544488 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:35:47.544747 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:35:47.545587 (+ 840us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 11:35:47.545591 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:35:47.545637 (+ 46us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 11:35:47.545741 (+ 104us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:35:47.546249 (+ 508us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:35:47.546259 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:35:47.546266 (+ 7us) client_negotiation.cc:769] Sending connection context
0810 11:35:47.546429 (+ 163us) client_negotiation.cc:240] Negotiation successful
0810 11:35:47.546916 (+ 487us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":545,"thread_start_us":340,"threads_started":1}
I20260810 11:35:47.550334 28899 heartbeater.cc:347] Connected to a master server at 127.8.18.254:42435
I20260810 11:35:47.550705 28899 heartbeater.cc:464] Registering TS with master...
I20260810 11:35:47.551530 28899 heartbeater.cc:511] Master 127.8.18.254:42435 requested a full tablet report, sending...
I20260810 11:35:47.554148 28701 ts_manager.cc:194] Registered new tserver with Master: c81ea21b94ba481fa3af5c88bc92d8c5 (127.8.18.193:45567)
I20260810 11:35:47.555835 28701 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.18.193:48379
W20260810 11:35:47.930563 28903 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:35:47.930881 28903 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:35:47.931041 28903 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:35:47.938803 28903 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:35:47.938933 28903 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:35:47.939105 28903 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.18.194
I20260810 11:35:47.952314 28903 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.18.212:35517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786361401922362-8267-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786361401922362-8267-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.18.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786361401922362-8267-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.18.194
--webserver_port=0
--tserver_master_addrs=127.8.18.254:42435
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.18.194
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786361401922362-8267-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:35:47.954115 28903 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:35:47.956827 28903 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:35:47.971565 28909 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:35:47.971606 28910 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:35:47.971643 28912 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:35:47.972533 28903 server_base.cc:1034] running on GCE node
I20260810 11:35:47.973277 28903 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:35:47.974396 28903 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:35:47.975625 28903 hybrid_clock.cc:648] HybridClock initialized: now 1786361747975569 us; error 51 us; skew 500 ppm
I20260810 11:35:47.975986 28903 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 11:35:47.978644 28903 webserver.cc:460] Webserver started at http://127.8.18.194:37109/ using document root <none> and password file <none>
I20260810 11:35:47.979417 28903 fs_manager.cc:359] Metadata directory not provided
I20260810 11:35:47.979542 28903 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:35:47.979907 28903 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:35:47.983271 28903 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786361401922362-8267-0/minicluster-data/ts-1/data/instance:
uuid: "44b0639c158a4faf9faf36e1a99af6f1"
format_stamp: "Formatted at 2026-08-10 11:35:47 on dist-test-slave-srgv"
server_key: "83f8d253da3c73ce1e9289f813ec1fc9"
server_key_iv: "ded786be48fc8af3eab258bd9a48dc6e"
server_key_version: "encryptionkey@0"
I20260810 11:35:47.984196 28903 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786361401922362-8267-0/minicluster-data/ts-1/wal/instance:
uuid: "44b0639c158a4faf9faf36e1a99af6f1"
format_stamp: "Formatted at 2026-08-10 11:35:47 on dist-test-slave-srgv"
server_key: "83f8d253da3c73ce1e9289f813ec1fc9"
server_key_iv: "ded786be48fc8af3eab258bd9a48dc6e"
server_key_version: "encryptionkey@0"
I20260810 11:35:47.990206 28903 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260810 11:35:47.994622 28918 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:35:47.996268 28903 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260810 11:35:47.996495 28903 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786361401922362-8267-0/minicluster-data/ts-1/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786361401922362-8267-0/minicluster-data/ts-1/wal
uuid: "44b0639c158a4faf9faf36e1a99af6f1"
format_stamp: "Formatted at 2026-08-10 11:35:47 on dist-test-slave-srgv"
server_key: "83f8d253da3c73ce1e9289f813ec1fc9"
server_key_iv: "ded786be48fc8af3eab258bd9a48dc6e"
server_key_version: "encryptionkey@0"
I20260810 11:35:47.996755 28903 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786361401922362-8267-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786361401922362-8267-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786361401922362-8267-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:35:48.015000 28903 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:35:48.015967 28903 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:35:48.016338 28903 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:35:48.017496 28903 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:35:48.019099 28903 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:35:48.019236 28903 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:35:48.019378 28903 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:35:48.019451 28903 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:35:48.055601 28903 rpc_server.cc:307] RPC server started. Bound to: 127.8.18.194:32981
I20260810 11:35:48.055694 29034 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.18.194:32981 every 8 connection(s)
I20260810 11:35:48.057133 28903 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786361401922362-8267-0/minicluster-data/ts-1/data/info.pb
I20260810 11:35:48.064872 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 28903
I20260810 11:35:48.065021 8267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786361401922362-8267-0/minicluster-data/ts-1/wal/instance
I20260810 11:35:48.065697 8267 external_mini_cluster.cc:1428] Setting key a9d2f879f01659e434b8a3d239c635e3
I20260810 11:35:48.068979 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786361401922362-8267-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786361401922362-8267-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786361401922362-8267-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786361401922362-8267-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.18.195:0
--local_ip_for_outbound_sockets=127.8.18.195
--webserver_interface=127.8.18.195
--webserver_port=0
--tserver_master_addrs=127.8.18.254:42435
--builtin_ntp_servers=127.8.18.212:35517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260810 11:35:48.071843 29037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:48.059937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.194:40427 (local address 127.8.18.254:42435)
0810 11:35:48.060423 (+ 486us) server_negotiation.cc:206] Beginning negotiation
0810 11:35:48.060430 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:35:48.061848 (+ 1418us) server_negotiation.cc:407] Connection header received
0810 11:35:48.063739 (+ 1891us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:35:48.063744 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:35:48.063821 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:35:48.063929 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:35:48.065303 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:48.066423 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:48.068512 (+ 2089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:48.069065 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:48.069238 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:35:48.070959 (+ 1721us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:35:48.070965 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:35:48.070968 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:35:48.070988 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:35:48.071032 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:35:48.071045 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:35:48.071141 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0810 11:35:48.071374 (+ 233us) server_negotiation.cc:299] Negotiation successful
0810 11:35:48.071489 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":355,"thread_start_us":222,"threads_started":1}
I20260810 11:35:48.072927 29038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:48.060579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:42435 (local address 127.8.18.194:40427)
0810 11:35:48.061677 (+ 1098us) negotiation.cc:107] Waiting for socket to connect
0810 11:35:48.061726 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0810 11:35:48.063431 (+ 1705us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:35:48.064196 (+ 765us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:35:48.064205 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:35:48.064300 (+ 95us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:35:48.065033 (+ 733us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:35:48.065064 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:35:48.066607 (+ 1543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:35:48.066618 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:35:48.068320 (+ 1702us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:35:48.068336 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:35:48.069523 (+ 1187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:35:48.069529 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:35:48.069761 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:35:48.070590 (+ 829us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 11:35:48.070592 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:35:48.070637 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 11:35:48.070744 (+ 107us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:35:48.071141 (+ 397us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:35:48.071157 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:35:48.071167 (+ 10us) client_negotiation.cc:769] Sending connection context
0810 11:35:48.071361 (+ 194us) client_negotiation.cc:240] Negotiation successful
0810 11:35:48.071834 (+ 473us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":530,"thread_start_us":319,"threads_started":1}
I20260810 11:35:48.074525 29035 heartbeater.cc:347] Connected to a master server at 127.8.18.254:42435
I20260810 11:35:48.074869 29035 heartbeater.cc:464] Registering TS with master...
I20260810 11:35:48.075676 29035 heartbeater.cc:511] Master 127.8.18.254:42435 requested a full tablet report, sending...
I20260810 11:35:48.077311 28701 ts_manager.cc:194] Registered new tserver with Master: 44b0639c158a4faf9faf36e1a99af6f1 (127.8.18.194:32981)
I20260810 11:35:48.078615 28701 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.18.194:40427
W20260810 11:35:48.444170 29039 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:35:48.444501 29039 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:35:48.444700 29039 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:35:48.453084 29039 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:35:48.453210 29039 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:35:48.453446 29039 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.18.195
I20260810 11:35:48.464675 29039 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.18.212:35517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786361401922362-8267-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786361401922362-8267-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.18.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786361401922362-8267-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.18.195
--webserver_port=0
--tserver_master_addrs=127.8.18.254:42435
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.18.195
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786361401922362-8267-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:35:48.466209 29039 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:35:48.468225 29039 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:35:48.483773 29047 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:35:48.483752 29045 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:35:48.483819 29044 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:35:48.484655 29039 server_base.cc:1034] running on GCE node
I20260810 11:35:48.485450 29039 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:35:48.486658 29039 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:35:48.487937 29039 hybrid_clock.cc:648] HybridClock initialized: now 1786361748487879 us; error 75 us; skew 500 ppm
I20260810 11:35:48.488384 29039 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 11:35:48.491225 29039 webserver.cc:460] Webserver started at http://127.8.18.195:42875/ using document root <none> and password file <none>
I20260810 11:35:48.492062 29039 fs_manager.cc:359] Metadata directory not provided
I20260810 11:35:48.492211 29039 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:35:48.492568 29039 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:35:48.496043 29039 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786361401922362-8267-0/minicluster-data/ts-2/data/instance:
uuid: "bd59dac9bda540e6a7330186ae2ea255"
format_stamp: "Formatted at 2026-08-10 11:35:48 on dist-test-slave-srgv"
server_key: "9e7e4505b74a399f15f2b4213ff8f11e"
server_key_iv: "3f0699f2e3175bcef12657d786374f75"
server_key_version: "encryptionkey@0"
I20260810 11:35:48.497052 29039 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786361401922362-8267-0/minicluster-data/ts-2/wal/instance:
uuid: "bd59dac9bda540e6a7330186ae2ea255"
format_stamp: "Formatted at 2026-08-10 11:35:48 on dist-test-slave-srgv"
server_key: "9e7e4505b74a399f15f2b4213ff8f11e"
server_key_iv: "3f0699f2e3175bcef12657d786374f75"
server_key_version: "encryptionkey@0"
I20260810 11:35:48.503110 29039 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.000s
I20260810 11:35:48.507112 29053 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:35:48.508823 29039 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 11:35:48.509079 29039 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786361401922362-8267-0/minicluster-data/ts-2/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786361401922362-8267-0/minicluster-data/ts-2/wal
uuid: "bd59dac9bda540e6a7330186ae2ea255"
format_stamp: "Formatted at 2026-08-10 11:35:48 on dist-test-slave-srgv"
server_key: "9e7e4505b74a399f15f2b4213ff8f11e"
server_key_iv: "3f0699f2e3175bcef12657d786374f75"
server_key_version: "encryptionkey@0"
I20260810 11:35:48.509366 29039 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786361401922362-8267-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786361401922362-8267-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786361401922362-8267-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:35:48.541070 29039 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:35:48.542184 29039 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:35:48.542570 29039 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:35:48.543715 29039 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 11:35:48.545573 29039 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 11:35:48.545704 29039 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 11:35:48.545842 29039 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 11:35:48.545946 29039 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 11:35:48.560039 28899 heartbeater.cc:503] Master 127.8.18.254:42435 was elected leader, sending a full tablet report...
I20260810 11:35:48.585309 29039 rpc_server.cc:307] RPC server started. Bound to: 127.8.18.195:40961
I20260810 11:35:48.585371 29168 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.18.195:40961 every 8 connection(s)
I20260810 11:35:48.586745 29039 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786361401922362-8267-0/minicluster-data/ts-2/data/info.pb
I20260810 11:35:48.594391 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 29039
I20260810 11:35:48.594589 8267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786361401922362-8267-0/minicluster-data/ts-2/wal/instance
I20260810 11:35:48.595068 8267 external_mini_cluster.cc:1428] Setting key b4546f2f9d6013b53fd89e0b15d2db34
I20260810 11:35:48.601675 29171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:48.589662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.195:48613 (local address 127.8.18.254:42435)
0810 11:35:48.590145 (+ 483us) server_negotiation.cc:206] Beginning negotiation
0810 11:35:48.590150 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:35:48.591430 (+ 1280us) server_negotiation.cc:407] Connection header received
0810 11:35:48.593614 (+ 2184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:35:48.593620 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:35:48.593728 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:35:48.593857 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:35:48.595236 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:48.596231 (+ 995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:48.598672 (+ 2441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:48.599095 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:48.599210 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:35:48.600694 (+ 1484us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:35:48.600699 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:35:48.600702 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:35:48.600723 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:35:48.600772 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:35:48.600784 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:35:48.600888 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0810 11:35:48.601123 (+ 235us) server_negotiation.cc:299] Negotiation successful
0810 11:35:48.601224 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":372,"thread_start_us":267,"threads_started":1}
I20260810 11:35:48.602708 29172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:48.590199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:42435 (local address 127.8.18.195:48613)
0810 11:35:48.591240 (+ 1041us) negotiation.cc:107] Waiting for socket to connect
0810 11:35:48.591280 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0810 11:35:48.593234 (+ 1954us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:35:48.594136 (+ 902us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:35:48.594146 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:35:48.594258 (+ 112us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:35:48.595020 (+ 762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:35:48.595042 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:35:48.596496 (+ 1454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:35:48.596502 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:35:48.598521 (+ 2019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:35:48.598538 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:35:48.599320 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:35:48.599325 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:35:48.599541 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:35:48.600303 (+ 762us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 11:35:48.600306 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:35:48.600350 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 11:35:48.600463 (+ 113us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:35:48.600921 (+ 458us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:35:48.600930 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:35:48.600936 (+ 6us) client_negotiation.cc:769] Sending connection context
0810 11:35:48.601096 (+ 160us) client_negotiation.cc:240] Negotiation successful
0810 11:35:48.601590 (+ 494us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":549,"thread_start_us":339,"threads_started":1}
I20260810 11:35:48.604238 29169 heartbeater.cc:347] Connected to a master server at 127.8.18.254:42435
I20260810 11:35:48.604540 29169 heartbeater.cc:464] Registering TS with master...
I20260810 11:35:48.605322 29169 heartbeater.cc:511] Master 127.8.18.254:42435 requested a full tablet report, sending...
I20260810 11:35:48.606904 28701 ts_manager.cc:194] Registered new tserver with Master: bd59dac9bda540e6a7330186ae2ea255 (127.8.18.195:40961)
I20260810 11:35:48.607713 28701 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.18.195:48613
I20260810 11:35:48.610944 8267 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 11:35:48.621825 29171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:48.615473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60628 (local address 127.8.18.254:42435)
0810 11:35:48.615661 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0810 11:35:48.615666 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:35:48.616154 (+ 488us) server_negotiation.cc:407] Connection header received
0810 11:35:48.616287 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:35:48.616291 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:35:48.616359 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:35:48.616459 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:35:48.617111 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:48.617944 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:48.619397 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:48.619837 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:48.619946 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:35:48.621092 (+ 1146us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:35:48.621097 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:35:48.621100 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:35:48.621122 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:35:48.621193 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:35:48.621207 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:35:48.621283 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0810 11:35:48.621478 (+ 195us) server_negotiation.cc:299] Negotiation successful
0810 11:35:48.621570 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260810 11:35:48.626298 28701 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60628:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 11:35:48.633590 28701 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 11:35:48.663584 29182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:48.647136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.195:40961 (local address 127.0.0.1:41466)
0810 11:35:48.648317 (+ 1181us) negotiation.cc:107] Waiting for socket to connect
0810 11:35:48.648375 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0810 11:35:48.648600 (+ 225us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:35:48.649733 (+ 1133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:35:48.649742 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:35:48.649766 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:35:48.653611 (+ 3845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:35:48.653641 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:35:48.658187 (+ 4546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:35:48.658198 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:35:48.660897 (+ 2699us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:35:48.660910 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:35:48.662428 (+ 1518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:35:48.662434 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:35:48.662756 (+ 322us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:35:48.662811 (+ 55us) client_negotiation.cc:769] Sending connection context
0810 11:35:48.662981 (+ 170us) client_negotiation.cc:240] Negotiation successful
0810 11:35:48.663217 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":854,"mutex_wait_us":58,"thread_start_us":676,"threads_started":1}
I20260810 11:35:48.664460 29183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:48.647480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41466 (local address 127.8.18.195:40961)
0810 11:35:48.648331 (+ 851us) server_negotiation.cc:206] Beginning negotiation
0810 11:35:48.648340 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 11:35:48.648779 (+ 439us) server_negotiation.cc:407] Connection header received
0810 11:35:48.648912 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:35:48.648924 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:35:48.649205 (+ 281us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:35:48.649401 (+ 196us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:35:48.656162 (+ 6761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:48.657942 (+ 1780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:48.661151 (+ 3209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:48.662198 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:48.662380 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:35:48.662719 (+ 339us) server_negotiation.cc:1035] Waiting for connection context
0810 11:35:48.663058 (+ 339us) server_negotiation.cc:299] Negotiation successful
0810 11:35:48.663276 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":678,"thread_start_us":497,"threads_started":1}
I20260810 11:35:48.665699 29184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:48.648656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.194:32981 (local address 127.0.0.1:57976)
0810 11:35:48.649290 (+ 634us) negotiation.cc:107] Waiting for socket to connect
0810 11:35:48.649312 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 11:35:48.649451 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:35:48.657758 (+ 8307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:35:48.657763 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:35:48.657791 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:35:48.658379 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:35:48.658395 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:35:48.660182 (+ 1787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:35:48.660191 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:35:48.662719 (+ 2528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:35:48.662735 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:35:48.664797 (+ 2062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:35:48.664805 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:35:48.664988 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:35:48.665033 (+ 45us) client_negotiation.cc:769] Sending connection context
0810 11:35:48.665122 (+ 89us) client_negotiation.cc:240] Negotiation successful
0810 11:35:48.665269 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":492,"mutex_wait_us":92,"thread_start_us":336,"threads_started":1}
I20260810 11:35:48.666254 29186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:48.650294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.193:45567 (local address 127.0.0.1:60034)
0810 11:35:48.656624 (+ 6330us) negotiation.cc:107] Waiting for socket to connect
0810 11:35:48.656645 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 11:35:48.656886 (+ 241us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:35:48.658077 (+ 1191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:35:48.658081 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:35:48.658109 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:35:48.658699 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:35:48.658711 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:35:48.660732 (+ 2021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:35:48.660739 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:35:48.663048 (+ 2309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:35:48.663062 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:35:48.665378 (+ 2316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:35:48.665383 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:35:48.665643 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:35:48.665675 (+ 32us) client_negotiation.cc:769] Sending connection context
0810 11:35:48.665772 (+ 97us) client_negotiation.cc:240] Negotiation successful
0810 11:35:48.665906 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6179,"mutex_wait_us":51,"thread_start_us":6012,"threads_started":1}
I20260810 11:35:48.666441 29185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:48.648754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57976 (local address 127.8.18.194:32981)
0810 11:35:48.657011 (+ 8257us) server_negotiation.cc:206] Beginning negotiation
0810 11:35:48.657020 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 11:35:48.657053 (+ 33us) server_negotiation.cc:407] Connection header received
0810 11:35:48.657157 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:35:48.657165 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:35:48.657510 (+ 345us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:35:48.657760 (+ 250us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:35:48.658570 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:48.660033 (+ 1463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:48.662947 (+ 2914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:48.664545 (+ 1598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:48.665575 (+ 1030us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:35:48.665830 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0810 11:35:48.665904 (+ 74us) server_negotiation.cc:299] Negotiation successful
0810 11:35:48.666093 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8108,"thread_start_us":7997,"threads_started":1}
I20260810 11:35:48.667268 29187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:48.650623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60034 (local address 127.8.18.193:45567)
0810 11:35:48.656116 (+ 5493us) server_negotiation.cc:206] Beginning negotiation
0810 11:35:48.656125 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 11:35:48.656760 (+ 635us) server_negotiation.cc:407] Connection header received
0810 11:35:48.657164 (+ 404us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:35:48.657174 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:35:48.657509 (+ 335us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:35:48.657766 (+ 257us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:35:48.658937 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:48.660565 (+ 1628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:48.663290 (+ 2725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:48.664829 (+ 1539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:48.664968 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:35:48.665262 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0810 11:35:48.666626 (+ 1364us) server_negotiation.cc:299] Negotiation successful
0810 11:35:48.666823 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5306,"thread_start_us":5161,"threads_started":1}
I20260810 11:35:48.669234 29101 tablet_service.cc:1467] Processing CreateTablet for tablet 3e4246bf9ec64c1d8bed41cc9a8cfd4d (DEFAULT_TABLE table=test-table [id=45dcdd9c1dbf4e7f9b07987f43b0b5a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:35:48.670984 29101 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e4246bf9ec64c1d8bed41cc9a8cfd4d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:35:48.671233 28829 tablet_service.cc:1467] Processing CreateTablet for tablet 3e4246bf9ec64c1d8bed41cc9a8cfd4d (DEFAULT_TABLE table=test-table [id=45dcdd9c1dbf4e7f9b07987f43b0b5a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:35:48.671233 28966 tablet_service.cc:1467] Processing CreateTablet for tablet 3e4246bf9ec64c1d8bed41cc9a8cfd4d (DEFAULT_TABLE table=test-table [id=45dcdd9c1dbf4e7f9b07987f43b0b5a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 11:35:48.672778 28829 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e4246bf9ec64c1d8bed41cc9a8cfd4d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:35:48.672776 28966 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e4246bf9ec64c1d8bed41cc9a8cfd4d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:35:48.683777 29188 tablet_bootstrap.cc:492] T 3e4246bf9ec64c1d8bed41cc9a8cfd4d P bd59dac9bda540e6a7330186ae2ea255: Bootstrap starting.
I20260810 11:35:48.687824 29188 tablet_bootstrap.cc:654] T 3e4246bf9ec64c1d8bed41cc9a8cfd4d P bd59dac9bda540e6a7330186ae2ea255: Neither blocks nor log segments found. Creating new log.
I20260810 11:35:48.689559 29190 tablet_bootstrap.cc:492] T 3e4246bf9ec64c1d8bed41cc9a8cfd4d P 44b0639c158a4faf9faf36e1a99af6f1: Bootstrap starting.
I20260810 11:35:48.689536 29188 log.cc:824] T 3e4246bf9ec64c1d8bed41cc9a8cfd4d P bd59dac9bda540e6a7330186ae2ea255: Log is configured to *not* fsync() on all Append() calls
I20260810 11:35:48.692173 29189 tablet_bootstrap.cc:492] T 3e4246bf9ec64c1d8bed41cc9a8cfd4d P c81ea21b94ba481fa3af5c88bc92d8c5: Bootstrap starting.
I20260810 11:35:48.694072 29190 tablet_bootstrap.cc:654] T 3e4246bf9ec64c1d8bed41cc9a8cfd4d P 44b0639c158a4faf9faf36e1a99af6f1: Neither blocks nor log segments found. Creating new log.
I20260810 11:35:48.695544 29190 log.cc:824] T 3e4246bf9ec64c1d8bed41cc9a8cfd4d P 44b0639c158a4faf9faf36e1a99af6f1: Log is configured to *not* fsync() on all Append() calls
I20260810 11:35:48.696352 29189 tablet_bootstrap.cc:654] T 3e4246bf9ec64c1d8bed41cc9a8cfd4d P c81ea21b94ba481fa3af5c88bc92d8c5: Neither blocks nor log segments found. Creating new log.
I20260810 11:35:48.697765 29189 log.cc:824] T 3e4246bf9ec64c1d8bed41cc9a8cfd4d P c81ea21b94ba481fa3af5c88bc92d8c5: Log is configured to *not* fsync() on all Append() calls
I20260810 11:35:48.698830 29190 tablet_bootstrap.cc:492] T 3e4246bf9ec64c1d8bed41cc9a8cfd4d P 44b0639c158a4faf9faf36e1a99af6f1: No bootstrap required, opened a new log
I20260810 11:35:48.698875 29188 tablet_bootstrap.cc:492] T 3e4246bf9ec64c1d8bed41cc9a8cfd4d P bd59dac9bda540e6a7330186ae2ea255: No bootstrap required, opened a new log
I20260810 11:35:48.699160 29190 ts_tablet_manager.cc:1397] T 3e4246bf9ec64c1d8bed41cc9a8cfd4d P 44b0639c158a4faf9faf36e1a99af6f1: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260810 11:35:48.699246 29188 ts_tablet_manager.cc:1397] T 3e4246bf9ec64c1d8bed41cc9a8cfd4d P bd59dac9bda540e6a7330186ae2ea255: Time spent bootstrapping tablet: real 0.016s user 0.005s sys 0.004s
I20260810 11:35:48.707664 29190 raft_consensus.cc:348] T 3e4246bf9ec64c1d8bed41cc9a8cfd4d P 44b0639c158a4faf9faf36e1a99af6f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd59dac9bda540e6a7330186ae2ea255" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 40961 } } peers { permanent_uuid: "44b0639c158a4faf9faf36e1a99af6f1" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 32981 } } peers { permanent_uuid: "c81ea21b94ba481fa3af5c88bc92d8c5" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 45567 } }
I20260810 11:35:48.708122 29190 raft_consensus.cc:374] T 3e4246bf9ec64c1d8bed41cc9a8cfd4d P 44b0639c158a4faf9faf36e1a99af6f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:35:48.708228 29190 raft_consensus.cc:729] T 3e4246bf9ec64c1d8bed41cc9a8cfd4d P 44b0639c158a4faf9faf36e1a99af6f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44b0639c158a4faf9faf36e1a99af6f1, State: Initialized, Role: FOLLOWER
I20260810 11:35:48.708374 29189 tablet_bootstrap.cc:492] T 3e4246bf9ec64c1d8bed41cc9a8cfd4d P c81ea21b94ba481fa3af5c88bc92d8c5: No bootstrap required, opened a new log
I20260810 11:35:48.708751 29189 ts_tablet_manager.cc:1397] T 3e4246bf9ec64c1d8bed41cc9a8cfd4d P c81ea21b94ba481fa3af5c88bc92d8c5: Time spent bootstrapping tablet: real 0.017s user 0.004s sys 0.009s
I20260810 11:35:48.709029 29190 consensus_queue.cc:260] T 3e4246bf9ec64c1d8bed41cc9a8cfd4d P 44b0639c158a4faf9faf36e1a99af6f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd59dac9bda540e6a7330186ae2ea255" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 40961 } } peers { permanent_uuid: "44b0639c158a4faf9faf36e1a99af6f1" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 32981 } } peers { permanent_uuid: "c81ea21b94ba481fa3af5c88bc92d8c5" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 45567 } }
I20260810 11:35:48.710369 29035 heartbeater.cc:503] Master 127.8.18.254:42435 was elected leader, sending a full tablet report...
I20260810 11:35:48.711037 29190 ts_tablet_manager.cc:1428] T 3e4246bf9ec64c1d8bed41cc9a8cfd4d P 44b0639c158a4faf9faf36e1a99af6f1: Time spent starting tablet: real 0.012s user 0.012s sys 0.000s
I20260810 11:35:48.712643 29188 raft_consensus.cc:348] T 3e4246bf9ec64c1d8bed41cc9a8cfd4d P bd59dac9bda540e6a7330186ae2ea255 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd59dac9bda540e6a7330186ae2ea255" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 40961 } } peers { permanent_uuid: "44b0639c158a4faf9faf36e1a99af6f1" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 32981 } } peers { permanent_uuid: "c81ea21b94ba481fa3af5c88bc92d8c5" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 45567 } }
I20260810 11:35:48.713264 29188 raft_consensus.cc:374] T 3e4246bf9ec64c1d8bed41cc9a8cfd4d P bd59dac9bda540e6a7330186ae2ea255 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:35:48.713477 29188 raft_consensus.cc:729] T 3e4246bf9ec64c1d8bed41cc9a8cfd4d P bd59dac9bda540e6a7330186ae2ea255 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd59dac9bda540e6a7330186ae2ea255, State: Initialized, Role: FOLLOWER
I20260810 11:35:48.714325 29188 consensus_queue.cc:260] T 3e4246bf9ec64c1d8bed41cc9a8cfd4d P bd59dac9bda540e6a7330186ae2ea255 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd59dac9bda540e6a7330186ae2ea255" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 40961 } } peers { permanent_uuid: "44b0639c158a4faf9faf36e1a99af6f1" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 32981 } } peers { permanent_uuid: "c81ea21b94ba481fa3af5c88bc92d8c5" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 45567 } }
I20260810 11:35:48.722348 29169 heartbeater.cc:503] Master 127.8.18.254:42435 was elected leader, sending a full tablet report...
I20260810 11:35:48.723364 29188 ts_tablet_manager.cc:1428] T 3e4246bf9ec64c1d8bed41cc9a8cfd4d P bd59dac9bda540e6a7330186ae2ea255: Time spent starting tablet: real 0.024s user 0.020s sys 0.004s
I20260810 11:35:48.724411 29189 raft_consensus.cc:348] T 3e4246bf9ec64c1d8bed41cc9a8cfd4d P c81ea21b94ba481fa3af5c88bc92d8c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd59dac9bda540e6a7330186ae2ea255" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 40961 } } peers { permanent_uuid: "44b0639c158a4faf9faf36e1a99af6f1" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 32981 } } peers { permanent_uuid: "c81ea21b94ba481fa3af5c88bc92d8c5" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 45567 } }
I20260810 11:35:48.724886 29189 raft_consensus.cc:374] T 3e4246bf9ec64c1d8bed41cc9a8cfd4d P c81ea21b94ba481fa3af5c88bc92d8c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:35:48.725010 29189 raft_consensus.cc:729] T 3e4246bf9ec64c1d8bed41cc9a8cfd4d P c81ea21b94ba481fa3af5c88bc92d8c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c81ea21b94ba481fa3af5c88bc92d8c5, State: Initialized, Role: FOLLOWER
I20260810 11:35:48.725708 29189 consensus_queue.cc:260] T 3e4246bf9ec64c1d8bed41cc9a8cfd4d P c81ea21b94ba481fa3af5c88bc92d8c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd59dac9bda540e6a7330186ae2ea255" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 40961 } } peers { permanent_uuid: "44b0639c158a4faf9faf36e1a99af6f1" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 32981 } } peers { permanent_uuid: "c81ea21b94ba481fa3af5c88bc92d8c5" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 45567 } }
I20260810 11:35:48.728716 29189 ts_tablet_manager.cc:1428] T 3e4246bf9ec64c1d8bed41cc9a8cfd4d P c81ea21b94ba481fa3af5c88bc92d8c5: Time spent starting tablet: real 0.020s user 0.016s sys 0.003s
W20260810 11:35:48.784510 28900 tablet.cc:2369] T 3e4246bf9ec64c1d8bed41cc9a8cfd4d P c81ea21b94ba481fa3af5c88bc92d8c5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 11:35:48.798919 29195 raft_consensus.cc:482] T 3e4246bf9ec64c1d8bed41cc9a8cfd4d P bd59dac9bda540e6a7330186ae2ea255 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 11:35:48.799347 29195 raft_consensus.cc:504] T 3e4246bf9ec64c1d8bed41cc9a8cfd4d P bd59dac9bda540e6a7330186ae2ea255 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd59dac9bda540e6a7330186ae2ea255" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 40961 } } peers { permanent_uuid: "44b0639c158a4faf9faf36e1a99af6f1" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 32981 } } peers { permanent_uuid: "c81ea21b94ba481fa3af5c88bc92d8c5" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 45567 } }
I20260810 11:35:48.801295 29195 leader_election.cc:288] T 3e4246bf9ec64c1d8bed41cc9a8cfd4d P bd59dac9bda540e6a7330186ae2ea255 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 44b0639c158a4faf9faf36e1a99af6f1 (127.8.18.194:32981), c81ea21b94ba481fa3af5c88bc92d8c5 (127.8.18.193:45567)
I20260810 11:35:48.808614 29187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:48.801826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.195:55329 (local address 127.8.18.193:45567)
0810 11:35:48.802052 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0810 11:35:48.802060 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:35:48.802208 (+ 148us) server_negotiation.cc:407] Connection header received
0810 11:35:48.802523 (+ 315us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:35:48.802529 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:35:48.802643 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:35:48.802785 (+ 142us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:35:48.803434 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:48.804594 (+ 1160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:48.806521 (+ 1927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:48.807604 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:48.807755 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:35:48.807810 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0810 11:35:48.808170 (+ 360us) server_negotiation.cc:299] Negotiation successful
0810 11:35:48.808306 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260810 11:35:48.808648 29172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:48.801674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.193:45567 (local address 127.8.18.195:55329)
0810 11:35:48.802062 (+ 388us) negotiation.cc:107] Waiting for socket to connect
0810 11:35:48.802089 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0810 11:35:48.802320 (+ 231us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:35:48.802857 (+ 537us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:35:48.802865 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:35:48.802901 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:35:48.803250 (+ 349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:35:48.803261 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:35:48.804736 (+ 1475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:35:48.804740 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:35:48.806304 (+ 1564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:35:48.806321 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:35:48.807780 (+ 1459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:35:48.807789 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:35:48.807979 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:35:48.808018 (+ 39us) client_negotiation.cc:769] Sending connection context
0810 11:35:48.808148 (+ 130us) client_negotiation.cc:240] Negotiation successful
0810 11:35:48.808328 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":211,"spinlock_wait_cycles":25984}
I20260810 11:35:48.809330 29185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:48.801819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.18.195:48087 (local address 127.8.18.194:32981)
0810 11:35:48.802057 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0810 11:35:48.802062 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:35:48.802887 (+ 825us) server_negotiation.cc:407] Connection header received
0810 11:35:48.803184 (+ 297us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:35:48.803190 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:35:48.803288 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:35:48.803380 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 11:35:48.804151 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:48.805086 (+ 935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:48.807204 (+ 2118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:48.808206 (+ 1002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:48.808362 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:35:48.808437 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0810 11:35:48.808840 (+ 403us) server_negotiation.cc:299] Negotiation successful
0810 11:35:48.809033 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
W20260810 11:35:48.809547 29036 tablet.cc:2369] T 3e4246bf9ec64c1d8bed41cc9a8cfd4d P 44b0639c158a4faf9faf36e1a99af6f1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 11:35:48.809916 29197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:48.801720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.194:32981 (local address 127.8.18.195:48087)
0810 11:35:48.802717 (+ 997us) negotiation.cc:107] Waiting for socket to connect
0810 11:35:48.802739 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 11:35:48.803024 (+ 285us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:35:48.803463 (+ 439us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:35:48.803467 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:35:48.803498 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 11:35:48.803908 (+ 410us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:35:48.803922 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:35:48.805254 (+ 1332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:35:48.805266 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:35:48.806965 (+ 1699us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:35:48.806984 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:35:48.808404 (+ 1420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:35:48.808411 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:35:48.808601 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:35:48.808648 (+ 47us) client_negotiation.cc:769] Sending connection context
0810 11:35:48.809426 (+ 778us) client_negotiation.cc:240] Negotiation successful
0810 11:35:48.809604 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":847,"thread_start_us":588,"threads_started":1}
I20260810 11:35:48.810038 28850 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e4246bf9ec64c1d8bed41cc9a8cfd4d" candidate_uuid: "bd59dac9bda540e6a7330186ae2ea255" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c81ea21b94ba481fa3af5c88bc92d8c5" is_pre_election: true
I20260810 11:35:48.810688 28850 raft_consensus.cc:2454] T 3e4246bf9ec64c1d8bed41cc9a8cfd4d P c81ea21b94ba481fa3af5c88bc92d8c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd59dac9bda540e6a7330186ae2ea255 in term 0.
I20260810 11:35:48.810930 28988 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e4246bf9ec64c1d8bed41cc9a8cfd4d" candidate_uuid: "bd59dac9bda540e6a7330186ae2ea255" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "44b0639c158a4faf9faf36e1a99af6f1" is_pre_election: true
I20260810 11:35:48.811445 29055 leader_election.cc:302] T 3e4246bf9ec64c1d8bed41cc9a8cfd4d P bd59dac9bda540e6a7330186ae2ea255 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd59dac9bda540e6a7330186ae2ea255, c81ea21b94ba481fa3af5c88bc92d8c5; no voters:
I20260810 11:35:48.811525 28988 raft_consensus.cc:2454] T 3e4246bf9ec64c1d8bed41cc9a8cfd4d P 44b0639c158a4faf9faf36e1a99af6f1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd59dac9bda540e6a7330186ae2ea255 in term 0.
I20260810 11:35:48.811861 29195 raft_consensus.cc:2781] T 3e4246bf9ec64c1d8bed41cc9a8cfd4d P bd59dac9bda540e6a7330186ae2ea255 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 11:35:48.812075 29195 raft_consensus.cc:482] T 3e4246bf9ec64c1d8bed41cc9a8cfd4d P bd59dac9bda540e6a7330186ae2ea255 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 11:35:48.812237 29195 raft_consensus.cc:3037] T 3e4246bf9ec64c1d8bed41cc9a8cfd4d P bd59dac9bda540e6a7330186ae2ea255 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:35:48.815088 29195 raft_consensus.cc:504] T 3e4246bf9ec64c1d8bed41cc9a8cfd4d P bd59dac9bda540e6a7330186ae2ea255 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd59dac9bda540e6a7330186ae2ea255" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 40961 } } peers { permanent_uuid: "44b0639c158a4faf9faf36e1a99af6f1" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 32981 } } peers { permanent_uuid: "c81ea21b94ba481fa3af5c88bc92d8c5" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 45567 } }
I20260810 11:35:48.815804 29195 leader_election.cc:288] T 3e4246bf9ec64c1d8bed41cc9a8cfd4d P bd59dac9bda540e6a7330186ae2ea255 [CANDIDATE]: Term 1 election: Requested vote from peers 44b0639c158a4faf9faf36e1a99af6f1 (127.8.18.194:32981), c81ea21b94ba481fa3af5c88bc92d8c5 (127.8.18.193:45567)
I20260810 11:35:48.816421 28850 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e4246bf9ec64c1d8bed41cc9a8cfd4d" candidate_uuid: "bd59dac9bda540e6a7330186ae2ea255" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c81ea21b94ba481fa3af5c88bc92d8c5"
I20260810 11:35:48.816624 28988 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e4246bf9ec64c1d8bed41cc9a8cfd4d" candidate_uuid: "bd59dac9bda540e6a7330186ae2ea255" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "44b0639c158a4faf9faf36e1a99af6f1"
I20260810 11:35:48.816756 28850 raft_consensus.cc:3037] T 3e4246bf9ec64c1d8bed41cc9a8cfd4d P c81ea21b94ba481fa3af5c88bc92d8c5 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:35:48.816902 28988 raft_consensus.cc:3037] T 3e4246bf9ec64c1d8bed41cc9a8cfd4d P 44b0639c158a4faf9faf36e1a99af6f1 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:35:48.819327 28988 raft_consensus.cc:2454] T 3e4246bf9ec64c1d8bed41cc9a8cfd4d P 44b0639c158a4faf9faf36e1a99af6f1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd59dac9bda540e6a7330186ae2ea255 in term 1.
I20260810 11:35:48.819330 28850 raft_consensus.cc:2454] T 3e4246bf9ec64c1d8bed41cc9a8cfd4d P c81ea21b94ba481fa3af5c88bc92d8c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd59dac9bda540e6a7330186ae2ea255 in term 1.
I20260810 11:35:48.819886 29056 leader_election.cc:302] T 3e4246bf9ec64c1d8bed41cc9a8cfd4d P bd59dac9bda540e6a7330186ae2ea255 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 44b0639c158a4faf9faf36e1a99af6f1, bd59dac9bda540e6a7330186ae2ea255; no voters:
I20260810 11:35:48.820290 29195 raft_consensus.cc:2781] T 3e4246bf9ec64c1d8bed41cc9a8cfd4d P bd59dac9bda540e6a7330186ae2ea255 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:35:48.820827 29195 raft_consensus.cc:686] T 3e4246bf9ec64c1d8bed41cc9a8cfd4d P bd59dac9bda540e6a7330186ae2ea255 [term 1 LEADER]: Becoming Leader. State: Replica: bd59dac9bda540e6a7330186ae2ea255, State: Running, Role: LEADER
I20260810 11:35:48.821368 29195 consensus_queue.cc:237] T 3e4246bf9ec64c1d8bed41cc9a8cfd4d P bd59dac9bda540e6a7330186ae2ea255 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd59dac9bda540e6a7330186ae2ea255" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 40961 } } peers { permanent_uuid: "44b0639c158a4faf9faf36e1a99af6f1" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 32981 } } peers { permanent_uuid: "c81ea21b94ba481fa3af5c88bc92d8c5" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 45567 } }
I20260810 11:35:48.826869 28701 catalog_manager.cc:5495] T 3e4246bf9ec64c1d8bed41cc9a8cfd4d P bd59dac9bda540e6a7330186ae2ea255 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd59dac9bda540e6a7330186ae2ea255 (127.8.18.195). New cstate: current_term: 1 leader_uuid: "bd59dac9bda540e6a7330186ae2ea255" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd59dac9bda540e6a7330186ae2ea255" member_type: VOTER last_known_addr { host: "127.8.18.195" port: 40961 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "44b0639c158a4faf9faf36e1a99af6f1" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 32981 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c81ea21b94ba481fa3af5c88bc92d8c5" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 45567 } health_report { overall_health: UNKNOWN } } }
W20260810 11:35:48.838676 29170 tablet.cc:2369] T 3e4246bf9ec64c1d8bed41cc9a8cfd4d P bd59dac9bda540e6a7330186ae2ea255: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 11:35:48.854317 29183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:48.848489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41472 (local address 127.8.18.195:40961)
0810 11:35:48.848685 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0810 11:35:48.848689 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 11:35:48.848797 (+ 108us) server_negotiation.cc:407] Connection header received
0810 11:35:48.849001 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:35:48.849005 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:35:48.849094 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:35:48.849225 (+ 131us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 11:35:48.849607 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:48.850381 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:48.851890 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:48.852403 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:48.852516 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:35:48.853051 (+ 535us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 11:35:48.853775 (+ 724us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 11:35:48.853908 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0810 11:35:48.853998 (+ 90us) server_negotiation.cc:299] Negotiation successful
0810 11:35:48.854076 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260810 11:35:48.863565 28850 raft_consensus.cc:1261] T 3e4246bf9ec64c1d8bed41cc9a8cfd4d P c81ea21b94ba481fa3af5c88bc92d8c5 [term 1 FOLLOWER]: Refusing update from remote peer bd59dac9bda540e6a7330186ae2ea255: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 11:35:48.863565 28988 raft_consensus.cc:1261] T 3e4246bf9ec64c1d8bed41cc9a8cfd4d P 44b0639c158a4faf9faf36e1a99af6f1 [term 1 FOLLOWER]: Refusing update from remote peer bd59dac9bda540e6a7330186ae2ea255: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 11:35:48.864702 29198 consensus_queue.cc:1035] T 3e4246bf9ec64c1d8bed41cc9a8cfd4d P bd59dac9bda540e6a7330186ae2ea255 [LEADER]: Connected to new peer: Peer: permanent_uuid: "44b0639c158a4faf9faf36e1a99af6f1" member_type: VOTER last_known_addr { host: "127.8.18.194" port: 32981 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 11:35:48.865105 29195 consensus_queue.cc:1035] T 3e4246bf9ec64c1d8bed41cc9a8cfd4d P bd59dac9bda540e6a7330186ae2ea255 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c81ea21b94ba481fa3af5c88bc92d8c5" member_type: VOTER last_known_addr { host: "127.8.18.193" port: 45567 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 11:35:48.878628 29202 mvcc.cc:205] Tried to move back new op lower bound from 7316937723334012928 to 7316937723176693760. Current Snapshot: MvccSnapshot[applied={T|T < 7316937723334012928}]
I20260810 11:35:48.906383 28701 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60628:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 11:35:48.906751 28701 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60628:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 11:35:48.911604 28701 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ea18459c3204475c9a59f534dc620eb8: Sending DeleteTablet for 3 replicas of tablet 3e4246bf9ec64c1d8bed41cc9a8cfd4d
I20260810 11:35:48.913097 28966 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e4246bf9ec64c1d8bed41cc9a8cfd4d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:35:48 UTC) from {username='slave'} at 127.0.0.1:57976
I20260810 11:35:48.913144 29101 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e4246bf9ec64c1d8bed41cc9a8cfd4d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:35:48 UTC) from {username='slave'} at 127.0.0.1:41466
I20260810 11:35:48.919061 28829 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e4246bf9ec64c1d8bed41cc9a8cfd4d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 11:35:48 UTC) from {username='slave'} at 127.0.0.1:60034
I20260810 11:35:48.924368 29211 tablet_replica.cc:330] T 3e4246bf9ec64c1d8bed41cc9a8cfd4d P bd59dac9bda540e6a7330186ae2ea255: stopping tablet replica
I20260810 11:35:48.925190 29211 raft_consensus.cc:2229] T 3e4246bf9ec64c1d8bed41cc9a8cfd4d P bd59dac9bda540e6a7330186ae2ea255 [term 1 LEADER]: Raft consensus shutting down.
I20260810 11:35:48.926229 29211 raft_consensus.cc:2258] T 3e4246bf9ec64c1d8bed41cc9a8cfd4d P bd59dac9bda540e6a7330186ae2ea255 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:35:48.929258 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 28763
I20260810 11:35:48.929272 29211 ts_tablet_manager.cc:1905] T 3e4246bf9ec64c1d8bed41cc9a8cfd4d P bd59dac9bda540e6a7330186ae2ea255: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:35:48.937386 29213 tablet_replica.cc:330] T 3e4246bf9ec64c1d8bed41cc9a8cfd4d P c81ea21b94ba481fa3af5c88bc92d8c5: stopping tablet replica
I20260810 11:35:48.938051 29213 raft_consensus.cc:2229] T 3e4246bf9ec64c1d8bed41cc9a8cfd4d P c81ea21b94ba481fa3af5c88bc92d8c5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 11:35:48.938721 29213 raft_consensus.cc:2258] T 3e4246bf9ec64c1d8bed41cc9a8cfd4d P c81ea21b94ba481fa3af5c88bc92d8c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:35:48.939500 29211 ts_tablet_manager.cc:1918] T 3e4246bf9ec64c1d8bed41cc9a8cfd4d P bd59dac9bda540e6a7330186ae2ea255: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 11:35:48.939751 29212 tablet_replica.cc:330] T 3e4246bf9ec64c1d8bed41cc9a8cfd4d P 44b0639c158a4faf9faf36e1a99af6f1: stopping tablet replica
I20260810 11:35:48.939779 29211 log.cc:1196] T 3e4246bf9ec64c1d8bed41cc9a8cfd4d P bd59dac9bda540e6a7330186ae2ea255: Deleting WAL directory at /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786361401922362-8267-0/minicluster-data/ts-2/wal/wals/3e4246bf9ec64c1d8bed41cc9a8cfd4d
I20260810 11:35:48.940280 29212 raft_consensus.cc:2229] T 3e4246bf9ec64c1d8bed41cc9a8cfd4d P 44b0639c158a4faf9faf36e1a99af6f1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 11:35:48.940668 29211 ts_tablet_manager.cc:1939] T 3e4246bf9ec64c1d8bed41cc9a8cfd4d P bd59dac9bda540e6a7330186ae2ea255: Deleting consensus metadata
I20260810 11:35:48.940824 29212 raft_consensus.cc:2258] T 3e4246bf9ec64c1d8bed41cc9a8cfd4d P 44b0639c158a4faf9faf36e1a99af6f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 11:35:48.942907 28685 catalog_manager.cc:4841] TS bd59dac9bda540e6a7330186ae2ea255 (127.8.18.195:40961): tablet 3e4246bf9ec64c1d8bed41cc9a8cfd4d (table test-table [id=45dcdd9c1dbf4e7f9b07987f43b0b5a3]) successfully deleted
I20260810 11:35:48.943320 29212 ts_tablet_manager.cc:1905] T 3e4246bf9ec64c1d8bed41cc9a8cfd4d P 44b0639c158a4faf9faf36e1a99af6f1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:35:48.945629 29213 ts_tablet_manager.cc:1905] T 3e4246bf9ec64c1d8bed41cc9a8cfd4d P c81ea21b94ba481fa3af5c88bc92d8c5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 11:35:48.947948 29187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:48.932690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60040 (local address 127.8.18.193:45567)
0810 11:35:48.932928 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0810 11:35:48.932938 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 11:35:48.938228 (+ 5290us) server_negotiation.cc:407] Connection header received
0810 11:35:48.938558 (+ 330us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:35:48.938567 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:35:48.938725 (+ 158us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:35:48.938857 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:35:48.939891 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:48.941006 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:48.943316 (+ 2310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:48.944117 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:48.944250 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:35:48.945929 (+ 1679us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:35:48.945941 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:35:48.945959 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:35:48.946830 (+ 871us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:35:48.946902 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:35:48.946934 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:35:48.947047 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0810 11:35:48.947367 (+ 320us) server_negotiation.cc:299] Negotiation successful
0810 11:35:48.947530 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260810 11:35:48.949303 28894 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:35:48.950743 29212 ts_tablet_manager.cc:1918] T 3e4246bf9ec64c1d8bed41cc9a8cfd4d P 44b0639c158a4faf9faf36e1a99af6f1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 11:35:48.950954 29212 log.cc:1196] T 3e4246bf9ec64c1d8bed41cc9a8cfd4d P 44b0639c158a4faf9faf36e1a99af6f1: Deleting WAL directory at /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786361401922362-8267-0/minicluster-data/ts-1/wal/wals/3e4246bf9ec64c1d8bed41cc9a8cfd4d
I20260810 11:35:48.951471 29212 ts_tablet_manager.cc:1939] T 3e4246bf9ec64c1d8bed41cc9a8cfd4d P 44b0639c158a4faf9faf36e1a99af6f1: Deleting consensus metadata
I20260810 11:35:48.953279 28687 catalog_manager.cc:4841] TS 44b0639c158a4faf9faf36e1a99af6f1 (127.8.18.194:32981): tablet 3e4246bf9ec64c1d8bed41cc9a8cfd4d (table test-table [id=45dcdd9c1dbf4e7f9b07987f43b0b5a3]) successfully deleted
I20260810 11:35:49.052809 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 28763
W20260810 11:35:49.081830 28686 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.18.193:45567 (error 108)
I20260810 11:35:49.082008 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 28903
I20260810 11:35:49.093784 29185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:49.083283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57980 (local address 127.8.18.194:32981)
0810 11:35:49.083500 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0810 11:35:49.083508 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:35:49.083830 (+ 322us) server_negotiation.cc:407] Connection header received
0810 11:35:49.084151 (+ 321us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:35:49.084158 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:35:49.084301 (+ 143us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:35:49.084397 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:35:49.085122 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:49.086522 (+ 1400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:49.088654 (+ 2132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:49.089390 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:49.089571 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:35:49.091394 (+ 1823us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:35:49.091406 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:35:49.091425 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:35:49.092333 (+ 908us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:35:49.092415 (+ 82us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:35:49.092450 (+ 35us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:35:49.092569 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0810 11:35:49.092869 (+ 300us) server_negotiation.cc:299] Negotiation successful
0810 11:35:49.093017 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260810 11:35:49.095005 29030 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:35:49.095849 29184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:49.095143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.193:45567 (local address 127.0.0.1:60042)
0810 11:35:49.095389 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0810 11:35:49.095608 (+ 219us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.193:45567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260810 11:35:49.096400 28686 catalog_manager.cc:4584] TS c81ea21b94ba481fa3af5c88bc92d8c5 (127.8.18.193:45567): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3e4246bf9ec64c1d8bed41cc9a8cfd4d: Network error: Client connection negotiation failed: client connection to 127.8.18.193:45567: connect: Connection refused (error 111)
I20260810 11:35:49.150296 29184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:49.149697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.193:45567 (local address 127.0.0.1:60058)
0810 11:35:49.149959 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0810 11:35:49.150082 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.193:45567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260810 11:35:49.151726 29184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:49.151232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.193:45567 (local address 127.0.0.1:60064)
0810 11:35:49.151424 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0810 11:35:49.151541 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.193:45567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260810 11:35:49.205724 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 28903
I20260810 11:35:49.212692 29184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:49.212011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.193:45567 (local address 127.0.0.1:60080)
0810 11:35:49.212290 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0810 11:35:49.212449 (+ 159us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.193:45567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104}
I20260810 11:35:49.214108 29184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:49.213603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.193:45567 (local address 127.0.0.1:60088)
0810 11:35:49.213813 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0810 11:35:49.213939 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.193:45567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260810 11:35:49.228667 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 29039
I20260810 11:35:49.237561 29183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:49.229515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41478 (local address 127.8.18.195:40961)
0810 11:35:49.229745 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0810 11:35:49.229752 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:35:49.229776 (+ 24us) server_negotiation.cc:407] Connection header received
0810 11:35:49.229909 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:35:49.229918 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:35:49.230068 (+ 150us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:35:49.230192 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:35:49.230586 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:49.231823 (+ 1237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:49.233177 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:49.234159 (+ 982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:49.234321 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:35:49.235448 (+ 1127us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:35:49.235461 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:35:49.235480 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:35:49.236371 (+ 891us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:35:49.236441 (+ 70us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:35:49.236481 (+ 40us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:35:49.236611 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0810 11:35:49.236937 (+ 326us) server_negotiation.cc:299] Negotiation successful
0810 11:35:49.237069 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260810 11:35:49.238420 29164 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:35:49.295362 29184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:49.294745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.193:45567 (local address 127.0.0.1:60092)
0810 11:35:49.295012 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0810 11:35:49.295140 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.193:45567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96}
I20260810 11:35:49.296624 29184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:49.296154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.193:45567 (local address 127.0.0.1:60100)
0810 11:35:49.296390 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0810 11:35:49.296482 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.193:45567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122}
I20260810 11:35:49.368319 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 29039
I20260810 11:35:49.391474 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 28670
I20260810 11:35:49.392728 28733 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:35:49.510704 29184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:49.508748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.193:45567 (local address 127.0.0.1:60112)
0810 11:35:49.509726 (+ 978us) negotiation.cc:107] Waiting for socket to connect
0810 11:35:49.509825 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.193:45567: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":814}
I20260810 11:35:49.511165 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 28670
2026-08-10T11:35:49Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (3188 ms)
[----------] 6 tests from EncryptionPolicyTest (24197 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786361401922362-8267-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 11:35:49 dist-test-slave-srgv krb5kdc[29222](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 11:35:49 dist-test-slave-srgv krb5kdc[29222](info): set up 2 sockets
Aug 10 11:35:49 dist-test-slave-srgv krb5kdc[29222](info): commencing operation
krb5kdc: starting...
W20260810 11:35:50.681763 8267 mini_kdc.cc:121] Time spent starting KDC: real 1.123s user 0.002s sys 0.042s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786361401922362-8267-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786361401922362-8267-0/krb5kdc/test-user.keytab.
Aug 10 11:35:50 dist-test-slave-srgv krb5kdc[29222](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361750, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T11:35:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T11:35:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786361401922362-8267-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786361401922362-8267-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 11:35:51.017920 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786361401922362-8267-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786361401922362-8267-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786361401922362-8267-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39895
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786361401922362-8267-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37469
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39895
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786361401922362-8267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786361401922362-8267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786361401922362-8267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:35:51.383066 29235 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:35:51.383460 29235 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:35:51.383641 29235 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:35:51.391462 29235 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 11:35:51.391660 29235 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:35:51.391774 29235 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:35:51.391896 29235 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 11:35:51.391973 29235 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 11:35:51.402940 29235 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37469
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786361401922362-8267-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786361401922362-8267-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39895
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786361401922362-8267-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39895
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786361401922362-8267-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:35:51.404493 29235 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:35:51.406018 29235 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:35:51.419624 29240 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:35:51.419559 29241 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:35:51.419657 29243 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:35:51.420676 29235 server_base.cc:1034] running on GCE node
I20260810 11:35:51.421658 29235 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:35:51.423372 29235 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:35:51.424636 29235 hybrid_clock.cc:648] HybridClock initialized: now 1786361751424589 us; error 73 us; skew 500 ppm
I20260810 11:35:51.425030 29235 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:35:51 dist-test-slave-srgv krb5kdc[29222](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361751, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:35:51.434285 29235 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 11:35:51.436100 29235 webserver.cc:460] Webserver started at http://127.0.0.1:46609/ using document root <none> and password file <none>
I20260810 11:35:51.436906 29235 fs_manager.cc:359] Metadata directory not provided
I20260810 11:35:51.437016 29235 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:35:51.437322 29235 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:35:51.441051 29235 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786361401922362-8267-0/minicluster-data/master-0/data/instance:
uuid: "583f3c9abe134affb173fe733c5a3c95"
format_stamp: "Formatted at 2026-08-10 11:35:51 on dist-test-slave-srgv"
server_key: "a4b7ef27db903aae87f4d0f529dc8f72"
server_key_iv: "012bcb5d21d139cc45af345f76227fd1"
server_key_version: "encryptionkey@0"
I20260810 11:35:51.442006 29235 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786361401922362-8267-0/minicluster-data/master-0/wal/instance:
uuid: "583f3c9abe134affb173fe733c5a3c95"
format_stamp: "Formatted at 2026-08-10 11:35:51 on dist-test-slave-srgv"
server_key: "a4b7ef27db903aae87f4d0f529dc8f72"
server_key_iv: "012bcb5d21d139cc45af345f76227fd1"
server_key_version: "encryptionkey@0"
I20260810 11:35:51.448521 29235 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.001s sys 0.005s
I20260810 11:35:51.452461 29250 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:35:51.454074 29235 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 11:35:51.454325 29235 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786361401922362-8267-0/minicluster-data/master-0/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786361401922362-8267-0/minicluster-data/master-0/wal
uuid: "583f3c9abe134affb173fe733c5a3c95"
format_stamp: "Formatted at 2026-08-10 11:35:51 on dist-test-slave-srgv"
server_key: "a4b7ef27db903aae87f4d0f529dc8f72"
server_key_iv: "012bcb5d21d139cc45af345f76227fd1"
server_key_version: "encryptionkey@0"
I20260810 11:35:51.454568 29235 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786361401922362-8267-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786361401922362-8267-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786361401922362-8267-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:35:51.474331 29235 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:35:51.479140 29235 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:35:51.479488 29235 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:35:51.501024 29235 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39895
I20260810 11:35:51.501077 29305 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39895 every 8 connection(s)
I20260810 11:35:51.502559 29235 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
I20260810 11:35:51.506951 29306 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:35:51.512616 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 29235
I20260810 11:35:51.512909 8267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786361401922362-8267-0/minicluster-data/master-0/wal/instance
I20260810 11:35:51.513921 8267 external_mini_cluster.cc:1428] Setting key 8e9dc50df1ba1084addefadf03f6a558
I20260810 11:35:51.521200 29306 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 583f3c9abe134affb173fe733c5a3c95: Bootstrap starting.
I20260810 11:35:51.526067 29306 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 583f3c9abe134affb173fe733c5a3c95: Neither blocks nor log segments found. Creating new log.
I20260810 11:35:51.527631 29306 log.cc:824] T 00000000000000000000000000000000 P 583f3c9abe134affb173fe733c5a3c95: Log is configured to *not* fsync() on all Append() calls
Aug 10 11:35:51 dist-test-slave-srgv krb5kdc[29222](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361750, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260810 11:35:51.531895 29306 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 583f3c9abe134affb173fe733c5a3c95: No bootstrap required, opened a new log
I20260810 11:35:51.541921 29309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:51.516203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35069 (local address 127.0.0.1:39895)
0810 11:35:51.517126 (+ 923us) server_negotiation.cc:206] Beginning negotiation
0810 11:35:51.517139 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 11:35:51.517191 (+ 52us) server_negotiation.cc:407] Connection header received
0810 11:35:51.519163 (+ 1972us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:35:51.519191 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:35:51.519649 (+ 458us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:35:51.520007 (+ 358us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:35:51.522450 (+ 2443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:51.524143 (+ 1693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:51.526399 (+ 2256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:51.527178 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:51.527338 (+ 160us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:35:51.532975 (+ 5637us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:35:51.532986 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:35:51.533003 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:35:51.533056 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:35:51.536713 (+ 3657us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:35:51.537827 (+ 1114us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:35:51.537834 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:35:51.537845 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:35:51.538062 (+ 217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:35:51.538633 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:35:51.538637 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:35:51.538639 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:35:51.539330 (+ 691us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:35:51.539717 (+ 387us) server_negotiation.cc:1035] Waiting for connection context
0810 11:35:51.540301 (+ 584us) server_negotiation.cc:299] Negotiation successful
0810 11:35:51.540694 (+ 393us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":688,"thread_start_us":427,"threads_started":1}
I20260810 11:35:51.541617 29306 raft_consensus.cc:348] T 00000000000000000000000000000000 P 583f3c9abe134affb173fe733c5a3c95 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "583f3c9abe134affb173fe733c5a3c95" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39895 } }
I20260810 11:35:51.542156 29306 raft_consensus.cc:374] T 00000000000000000000000000000000 P 583f3c9abe134affb173fe733c5a3c95 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:35:51.542306 29306 raft_consensus.cc:729] T 00000000000000000000000000000000 P 583f3c9abe134affb173fe733c5a3c95 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 583f3c9abe134affb173fe733c5a3c95, State: Initialized, Role: FOLLOWER
I20260810 11:35:51.543162 29306 consensus_queue.cc:260] T 00000000000000000000000000000000 P 583f3c9abe134affb173fe733c5a3c95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "583f3c9abe134affb173fe733c5a3c95" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39895 } }
I20260810 11:35:51.543510 29306 raft_consensus.cc:388] T 00000000000000000000000000000000 P 583f3c9abe134affb173fe733c5a3c95 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 11:35:51.543673 29306 raft_consensus.cc:482] T 00000000000000000000000000000000 P 583f3c9abe134affb173fe733c5a3c95 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 11:35:51.543885 29306 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 583f3c9abe134affb173fe733c5a3c95 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:35:51.547009 8267 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 11:35:51.547312 29306 raft_consensus.cc:504] T 00000000000000000000000000000000 P 583f3c9abe134affb173fe733c5a3c95 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "583f3c9abe134affb173fe733c5a3c95" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39895 } }
I20260810 11:35:51.547920 29306 leader_election.cc:302] T 00000000000000000000000000000000 P 583f3c9abe134affb173fe733c5a3c95 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 583f3c9abe134affb173fe733c5a3c95; no voters:
I20260810 11:35:51.548689 29306 leader_election.cc:288] T 00000000000000000000000000000000 P 583f3c9abe134affb173fe733c5a3c95 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 11:35:51.548844 29311 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 583f3c9abe134affb173fe733c5a3c95 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:35:51.550302 29311 raft_consensus.cc:686] T 00000000000000000000000000000000 P 583f3c9abe134affb173fe733c5a3c95 [term 1 LEADER]: Becoming Leader. State: Replica: 583f3c9abe134affb173fe733c5a3c95, State: Running, Role: LEADER
I20260810 11:35:51.551018 29311 consensus_queue.cc:237] T 00000000000000000000000000000000 P 583f3c9abe134affb173fe733c5a3c95 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "583f3c9abe134affb173fe733c5a3c95" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39895 } }
I20260810 11:35:51.551594 29306 sys_catalog.cc:564] T 00000000000000000000000000000000 P 583f3c9abe134affb173fe733c5a3c95 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 11:35:51.555130 29312 sys_catalog.cc:455] T 00000000000000000000000000000000 P 583f3c9abe134affb173fe733c5a3c95 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "583f3c9abe134affb173fe733c5a3c95" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "583f3c9abe134affb173fe733c5a3c95" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39895 } } }
I20260810 11:35:51.555085 29313 sys_catalog.cc:455] T 00000000000000000000000000000000 P 583f3c9abe134affb173fe733c5a3c95 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 583f3c9abe134affb173fe733c5a3c95. Latest consensus state: current_term: 1 leader_uuid: "583f3c9abe134affb173fe733c5a3c95" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "583f3c9abe134affb173fe733c5a3c95" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39895 } } }
I20260810 11:35:51.555811 29312 sys_catalog.cc:458] T 00000000000000000000000000000000 P 583f3c9abe134affb173fe733c5a3c95 [sys.catalog]: This master's current role is: LEADER
I20260810 11:35:51.555809 29313 sys_catalog.cc:458] T 00000000000000000000000000000000 P 583f3c9abe134affb173fe733c5a3c95 [sys.catalog]: This master's current role is: LEADER
I20260810 11:35:51.557430 29322 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 11:35:51.569880 29322 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 11:35:51.584882 29322 catalog_manager.cc:1347] Generated new cluster ID: 440c3ede358e4544a281ae0b54019972
I20260810 11:35:51.585049 29322 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 11:35:51.610260 29309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:51.567174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39499 (local address 127.0.0.1:39895)
0810 11:35:51.567405 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0810 11:35:51.567415 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 11:35:51.578759 (+ 11344us) server_negotiation.cc:407] Connection header received
0810 11:35:51.579231 (+ 472us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:35:51.579241 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:35:51.579378 (+ 137us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:35:51.579492 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:35:51.584307 (+ 4815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:51.586303 (+ 1996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:51.589642 (+ 3339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:51.590754 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:51.590953 (+ 199us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:35:51.596608 (+ 5655us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:35:51.596636 (+ 28us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:35:51.596646 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:35:51.596728 (+ 82us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:35:51.602813 (+ 6085us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:35:51.604730 (+ 1917us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:35:51.604741 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:35:51.604747 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:35:51.605034 (+ 287us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:35:51.606081 (+ 1047us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:35:51.606091 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:35:51.606098 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:35:51.606978 (+ 880us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:35:51.607408 (+ 430us) server_negotiation.cc:1035] Waiting for connection context
0810 11:35:51.609180 (+ 1772us) server_negotiation.cc:299] Negotiation successful
0810 11:35:51.609547 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":46}
I20260810 11:35:51.610780 29322 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 11:35:51.612751 29322 catalog_manager.cc:1504] Loading token signing keys...
I20260810 11:35:51.625324 29322 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 583f3c9abe134affb173fe733c5a3c95: Generated new TSK 0
I20260810 11:35:51.626619 29322 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 11:35:51.636943 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 29235
I20260810 11:35:51.639761 29301 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:35:51.734215 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 29235
W20260810 11:35:51.753968 29220 connection.cc:535] client connection to 127.0.0.1:39895 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39895): Connection reset by peer (error 104)
2026-08-10T11:35:51Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (2216 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786361401922362-8267-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 11:35:51 dist-test-slave-srgv krb5kdc[29338](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 11:35:51 dist-test-slave-srgv krb5kdc[29338](info): set up 2 sockets
Aug 10 11:35:51 dist-test-slave-srgv krb5kdc[29338](info): commencing operation
krb5kdc: starting...
W20260810 11:35:52.912708 8267 mini_kdc.cc:121] Time spent starting KDC: real 1.139s user 0.007s sys 0.035s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786361401922362-8267-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786361401922362-8267-0/krb5kdc/test-user.keytab.
Aug 10 11:35:53 dist-test-slave-srgv krb5kdc[29338](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361753, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T11:35:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T11:35:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786361401922362-8267-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786361401922362-8267-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 11:35:53.244920 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786361401922362-8267-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786361401922362-8267-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786361401922362-8267-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38891
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786361401922362-8267-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35969
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38891
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786361401922362-8267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786361401922362-8267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786361401922362-8267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:35:53.616698 29351 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:35:53.617038 29351 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:35:53.617208 29351 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:35:53.625100 29351 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 11:35:53.625573 29351 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:35:53.625679 29351 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:35:53.625795 29351 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 11:35:53.625870 29351 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 11:35:53.636883 29351 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35969
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786361401922362-8267-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786361401922362-8267-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38891
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786361401922362-8267-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38891
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786361401922362-8267-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:35:53.638466 29351 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:35:53.639885 29351 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:35:53.652855 29359 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:35:53.652901 29356 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:35:53.652945 29357 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:35:53.653844 29351 server_base.cc:1034] running on GCE node
I20260810 11:35:53.654736 29351 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:35:53.656194 29351 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:35:53.657486 29351 hybrid_clock.cc:648] HybridClock initialized: now 1786361753657373 us; error 36 us; skew 500 ppm
I20260810 11:35:53.657878 29351 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:35:53 dist-test-slave-srgv krb5kdc[29338](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361753, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:35:53.661469 29351 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 11:35:53.663347 29351 webserver.cc:460] Webserver started at http://127.0.0.1:41993/ using document root <none> and password file <none>
I20260810 11:35:53.664168 29351 fs_manager.cc:359] Metadata directory not provided
I20260810 11:35:53.664315 29351 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:35:53.664693 29351 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:35:53.668267 29351 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786361401922362-8267-0/minicluster-data/master-0/data/instance:
uuid: "b2ce8e10254644d7a4f20b2b6bc654cd"
format_stamp: "Formatted at 2026-08-10 11:35:53 on dist-test-slave-srgv"
server_key: "a05c51718e0d3ab9f0b146f9dbbdde10"
server_key_iv: "7a7fc4818a9692bad3f56bd36a15685e"
server_key_version: "encryptionkey@0"
I20260810 11:35:53.669198 29351 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786361401922362-8267-0/minicluster-data/master-0/wal/instance:
uuid: "b2ce8e10254644d7a4f20b2b6bc654cd"
format_stamp: "Formatted at 2026-08-10 11:35:53 on dist-test-slave-srgv"
server_key: "a05c51718e0d3ab9f0b146f9dbbdde10"
server_key_iv: "7a7fc4818a9692bad3f56bd36a15685e"
server_key_version: "encryptionkey@0"
I20260810 11:35:53.675454 29351 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.000s
I20260810 11:35:53.679724 29366 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:35:53.681352 29351 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 11:35:53.681633 29351 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786361401922362-8267-0/minicluster-data/master-0/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786361401922362-8267-0/minicluster-data/master-0/wal
uuid: "b2ce8e10254644d7a4f20b2b6bc654cd"
format_stamp: "Formatted at 2026-08-10 11:35:53 on dist-test-slave-srgv"
server_key: "a05c51718e0d3ab9f0b146f9dbbdde10"
server_key_iv: "7a7fc4818a9692bad3f56bd36a15685e"
server_key_version: "encryptionkey@0"
I20260810 11:35:53.681870 29351 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786361401922362-8267-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786361401922362-8267-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786361401922362-8267-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:35:53.698273 29351 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:35:53.703114 29351 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:35:53.703460 29351 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:35:53.724664 29351 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38891
I20260810 11:35:53.724689 29418 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38891 every 8 connection(s)
I20260810 11:35:53.726269 29351 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
I20260810 11:35:53.729849 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 29351
I20260810 11:35:53.730077 8267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786361401922362-8267-0/minicluster-data/master-0/wal/instance
I20260810 11:35:53.730525 8267 external_mini_cluster.cc:1428] Setting key 8a767b5ba4271093da9b6cd3f197f43a
I20260810 11:35:53.730767 29419 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:35:53.747430 29419 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b2ce8e10254644d7a4f20b2b6bc654cd: Bootstrap starting.
I20260810 11:35:53.747699 29422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:53.732531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37509 (local address 127.0.0.1:38891)
0810 11:35:53.733460 (+ 929us) server_negotiation.cc:206] Beginning negotiation
0810 11:35:53.733476 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0810 11:35:53.733529 (+ 53us) server_negotiation.cc:407] Connection header received
0810 11:35:53.736064 (+ 2535us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:35:53.736100 (+ 36us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:35:53.736752 (+ 652us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:35:53.737295 (+ 543us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:35:53.737953 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:53.740153 (+ 2200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:53.741875 (+ 1722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:53.743051 (+ 1176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:53.743273 (+ 222us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:35:53.744530 (+ 1257us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:35:53.744541 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:35:53.744568 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:35:53.745396 (+ 828us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:35:53.745531 (+ 135us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:35:53.745566 (+ 35us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:35:53.745736 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0810 11:35:53.746085 (+ 349us) server_negotiation.cc:299] Negotiation successful
0810 11:35:53.746285 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":622,"thread_start_us":364,"threads_started":1}
I20260810 11:35:53.751653 29419 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b2ce8e10254644d7a4f20b2b6bc654cd: Neither blocks nor log segments found. Creating new log.
I20260810 11:35:53.752743 8267 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 11:35:53.753165 29419 log.cc:824] T 00000000000000000000000000000000 P b2ce8e10254644d7a4f20b2b6bc654cd: Log is configured to *not* fsync() on all Append() calls
I20260810 11:35:53.756742 29419 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b2ce8e10254644d7a4f20b2b6bc654cd: No bootstrap required, opened a new log
I20260810 11:35:53.765280 29422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:53.758131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34401 (local address 127.0.0.1:38891)
0810 11:35:53.758319 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0810 11:35:53.758324 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:35:53.758984 (+ 660us) server_negotiation.cc:407] Connection header received
0810 11:35:53.759291 (+ 307us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:35:53.759298 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:35:53.759366 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:35:53.759489 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:35:53.760056 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:53.760878 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:53.762445 (+ 1567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:53.762968 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:53.763122 (+ 154us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:35:53.764464 (+ 1342us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:35:53.764470 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:35:53.764473 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:35:53.764495 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:35:53.764540 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:35:53.764556 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:35:53.764691 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0810 11:35:53.764895 (+ 204us) server_negotiation.cc:299] Negotiation successful
0810 11:35:53.764954 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260810 11:35:53.765488 29419 raft_consensus.cc:348] T 00000000000000000000000000000000 P b2ce8e10254644d7a4f20b2b6bc654cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2ce8e10254644d7a4f20b2b6bc654cd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38891 } }
I20260810 11:35:53.766004 29419 raft_consensus.cc:374] T 00000000000000000000000000000000 P b2ce8e10254644d7a4f20b2b6bc654cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:35:53.766152 29419 raft_consensus.cc:729] T 00000000000000000000000000000000 P b2ce8e10254644d7a4f20b2b6bc654cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2ce8e10254644d7a4f20b2b6bc654cd, State: Initialized, Role: FOLLOWER
I20260810 11:35:53.766927 29419 consensus_queue.cc:260] T 00000000000000000000000000000000 P b2ce8e10254644d7a4f20b2b6bc654cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2ce8e10254644d7a4f20b2b6bc654cd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38891 } }
I20260810 11:35:53.767242 29419 raft_consensus.cc:388] T 00000000000000000000000000000000 P b2ce8e10254644d7a4f20b2b6bc654cd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 11:35:53.767379 29419 raft_consensus.cc:482] T 00000000000000000000000000000000 P b2ce8e10254644d7a4f20b2b6bc654cd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 11:35:53.767511 29419 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b2ce8e10254644d7a4f20b2b6bc654cd [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:35:53.769937 29419 raft_consensus.cc:504] T 00000000000000000000000000000000 P b2ce8e10254644d7a4f20b2b6bc654cd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2ce8e10254644d7a4f20b2b6bc654cd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38891 } }
I20260810 11:35:53.770524 29419 leader_election.cc:302] T 00000000000000000000000000000000 P b2ce8e10254644d7a4f20b2b6bc654cd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b2ce8e10254644d7a4f20b2b6bc654cd; no voters:
I20260810 11:35:53.771158 29419 leader_election.cc:288] T 00000000000000000000000000000000 P b2ce8e10254644d7a4f20b2b6bc654cd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 11:35:53.771343 29430 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b2ce8e10254644d7a4f20b2b6bc654cd [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:35:53.772054 29430 raft_consensus.cc:686] T 00000000000000000000000000000000 P b2ce8e10254644d7a4f20b2b6bc654cd [term 1 LEADER]: Becoming Leader. State: Replica: b2ce8e10254644d7a4f20b2b6bc654cd, State: Running, Role: LEADER
I20260810 11:35:53.772887 29430 consensus_queue.cc:237] T 00000000000000000000000000000000 P b2ce8e10254644d7a4f20b2b6bc654cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2ce8e10254644d7a4f20b2b6bc654cd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38891 } }
I20260810 11:35:53.774118 29419 sys_catalog.cc:564] T 00000000000000000000000000000000 P b2ce8e10254644d7a4f20b2b6bc654cd [sys.catalog]: configured and running, proceeding with master startup.
I20260810 11:35:53.776166 29431 sys_catalog.cc:455] T 00000000000000000000000000000000 P b2ce8e10254644d7a4f20b2b6bc654cd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b2ce8e10254644d7a4f20b2b6bc654cd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2ce8e10254644d7a4f20b2b6bc654cd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38891 } } }
I20260810 11:35:53.776733 29431 sys_catalog.cc:458] T 00000000000000000000000000000000 P b2ce8e10254644d7a4f20b2b6bc654cd [sys.catalog]: This master's current role is: LEADER
I20260810 11:35:53.776161 29432 sys_catalog.cc:455] T 00000000000000000000000000000000 P b2ce8e10254644d7a4f20b2b6bc654cd [sys.catalog]: SysCatalogTable state changed. Reason: New leader b2ce8e10254644d7a4f20b2b6bc654cd. Latest consensus state: current_term: 1 leader_uuid: "b2ce8e10254644d7a4f20b2b6bc654cd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2ce8e10254644d7a4f20b2b6bc654cd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38891 } } }
I20260810 11:35:53.778582 29432 sys_catalog.cc:458] T 00000000000000000000000000000000 P b2ce8e10254644d7a4f20b2b6bc654cd [sys.catalog]: This master's current role is: LEADER
I20260810 11:35:53.785660 29435 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 11:35:53.790729 29435 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 11:35:53.800794 29435 catalog_manager.cc:1347] Generated new cluster ID: 76880a3b55c04a40ad77b61203eed73e
I20260810 11:35:53.800951 29435 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 11:35:53.826272 29435 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 11:35:53.828377 29435 catalog_manager.cc:1504] Loading token signing keys...
I20260810 11:35:53.845945 29435 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b2ce8e10254644d7a4f20b2b6bc654cd: Generated new TSK 0
I20260810 11:35:53.847476 29435 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 11:35:53.860458 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 29351
W20260810 11:35:53.861708 29414 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:37509
W20260810 11:35:53.862675 8267 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 29351: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 11:35:53.862902 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 29351
W20260810 11:35:53.879961 29336 connection.cc:535] client connection to 127.0.0.1:38891 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38891): Connection reset by peer (error 104)
2026-08-10T11:35:53Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (2125 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786361401922362-8267-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 11:35:53 dist-test-slave-srgv krb5kdc[29452](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 11:35:53 dist-test-slave-srgv krb5kdc[29452](info): set up 2 sockets
Aug 10 11:35:53 dist-test-slave-srgv krb5kdc[29452](info): commencing operation
krb5kdc: starting...
W20260810 11:35:55.036928 8267 mini_kdc.cc:121] Time spent starting KDC: real 1.136s user 0.000s sys 0.044s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786361401922362-8267-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786361401922362-8267-0/krb5kdc/test-user.keytab.
Aug 10 11:35:55 dist-test-slave-srgv krb5kdc[29452](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361755, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T11:35:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T11:35:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786361401922362-8267-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786361401922362-8267-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 11:35:55.375070 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786361401922362-8267-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786361401922362-8267-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786361401922362-8267-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37163
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786361401922362-8267-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45385
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37163
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786361401922362-8267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786361401922362-8267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786361401922362-8267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:35:55.745012 29467 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:35:55.745354 29467 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:35:55.745538 29467 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:35:55.753396 29467 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 11:35:55.753569 29467 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:35:55.753661 29467 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:35:55.753774 29467 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 11:35:55.753850 29467 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 11:35:55.764444 29467 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45385
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786361401922362-8267-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786361401922362-8267-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37163
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786361401922362-8267-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37163
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786361401922362-8267-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:35:55.765985 29467 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:35:55.767443 29467 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:35:55.781446 29475 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:35:55.781481 29472 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:35:55.781519 29473 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:35:55.782473 29467 server_base.cc:1034] running on GCE node
I20260810 11:35:55.783392 29467 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:35:55.785115 29467 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:35:55.786386 29467 hybrid_clock.cc:648] HybridClock initialized: now 1786361755786308 us; error 73 us; skew 500 ppm
I20260810 11:35:55.786832 29467 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:35:55 dist-test-slave-srgv krb5kdc[29452](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361755, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:35:55.790565 29467 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 11:35:55.792258 29467 webserver.cc:460] Webserver started at http://127.0.0.1:43103/ using document root <none> and password file <none>
I20260810 11:35:55.793030 29467 fs_manager.cc:359] Metadata directory not provided
I20260810 11:35:55.793193 29467 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:35:55.793601 29467 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:35:55.797000 29467 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786361401922362-8267-0/minicluster-data/master-0/data/instance:
uuid: "5b7f7fd265134e5a885e0d46a95c7042"
format_stamp: "Formatted at 2026-08-10 11:35:55 on dist-test-slave-srgv"
server_key: "7fbb7ac2c740af759bf3a77c44408ab7"
server_key_iv: "4f4ab45b230cb9224d636d4ec57dee18"
server_key_version: "encryptionkey@0"
I20260810 11:35:55.797956 29467 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786361401922362-8267-0/minicluster-data/master-0/wal/instance:
uuid: "5b7f7fd265134e5a885e0d46a95c7042"
format_stamp: "Formatted at 2026-08-10 11:35:55 on dist-test-slave-srgv"
server_key: "7fbb7ac2c740af759bf3a77c44408ab7"
server_key_iv: "4f4ab45b230cb9224d636d4ec57dee18"
server_key_version: "encryptionkey@0"
I20260810 11:35:55.803927 29467 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260810 11:35:55.808075 29482 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:35:55.809732 29467 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260810 11:35:55.809984 29467 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786361401922362-8267-0/minicluster-data/master-0/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786361401922362-8267-0/minicluster-data/master-0/wal
uuid: "5b7f7fd265134e5a885e0d46a95c7042"
format_stamp: "Formatted at 2026-08-10 11:35:55 on dist-test-slave-srgv"
server_key: "7fbb7ac2c740af759bf3a77c44408ab7"
server_key_iv: "4f4ab45b230cb9224d636d4ec57dee18"
server_key_version: "encryptionkey@0"
I20260810 11:35:55.810231 29467 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786361401922362-8267-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786361401922362-8267-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786361401922362-8267-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:35:55.817035 29467 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:35:55.823254 29467 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:35:55.823639 29467 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:35:55.846926 29467 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37163
I20260810 11:35:55.846966 29535 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37163 every 8 connection(s)
I20260810 11:35:55.848552 29467 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
I20260810 11:35:55.850160 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 29467
I20260810 11:35:55.850356 8267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786361401922362-8267-0/minicluster-data/master-0/wal/instance
I20260810 11:35:55.850917 8267 external_mini_cluster.cc:1428] Setting key 559150e8ed6a855fb1d98d566e6aa09d
I20260810 11:35:55.853777 29536 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:35:55.860090 29539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:55.852959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52495 (local address 127.0.0.1:37163)
0810 11:35:55.853979 (+ 1020us) server_negotiation.cc:206] Beginning negotiation
0810 11:35:55.853993 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 11:35:55.854039 (+ 46us) server_negotiation.cc:407] Connection header received
0810 11:35:55.855945 (+ 1906us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:35:55.855972 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:35:55.856392 (+ 420us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:35:55.856886 (+ 494us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:35:55.858189 (+ 1303us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:35:55.858196 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:35:55.858214 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:35:55.858231 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:35:55.858308 (+ 77us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:35:55.858339 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:35:55.858634 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0810 11:35:55.858820 (+ 186us) server_negotiation.cc:299] Negotiation successful
0810 11:35:55.859039 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":713,"thread_start_us":443,"threads_started":1}
I20260810 11:35:55.864338 8267 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 11:35:55.866194 29536 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5b7f7fd265134e5a885e0d46a95c7042: Bootstrap starting.
I20260810 11:35:55.869928 29536 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5b7f7fd265134e5a885e0d46a95c7042: Neither blocks nor log segments found. Creating new log.
I20260810 11:35:55.871459 29536 log.cc:824] T 00000000000000000000000000000000 P 5b7f7fd265134e5a885e0d46a95c7042: Log is configured to *not* fsync() on all Append() calls
I20260810 11:35:55.872474 29539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:55.869309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55985 (local address 127.0.0.1:37163)
0810 11:35:55.869575 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0810 11:35:55.869583 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:35:55.869989 (+ 406us) server_negotiation.cc:407] Connection header received
0810 11:35:55.870257 (+ 268us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:35:55.870261 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:35:55.870314 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:35:55.870438 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:35:55.871580 (+ 1142us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:35:55.871590 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:35:55.871595 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:35:55.871621 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:35:55.871683 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:35:55.871710 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:35:55.871856 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0810 11:35:55.872002 (+ 146us) server_negotiation.cc:299] Negotiation successful
0810 11:35:55.872084 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117}
I20260810 11:35:55.875267 29536 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5b7f7fd265134e5a885e0d46a95c7042: No bootstrap required, opened a new log
I20260810 11:35:55.884415 29536 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5b7f7fd265134e5a885e0d46a95c7042 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b7f7fd265134e5a885e0d46a95c7042" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37163 } }
I20260810 11:35:55.884945 29536 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5b7f7fd265134e5a885e0d46a95c7042 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:35:55.885089 29536 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5b7f7fd265134e5a885e0d46a95c7042 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b7f7fd265134e5a885e0d46a95c7042, State: Initialized, Role: FOLLOWER
I20260810 11:35:55.885838 29536 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5b7f7fd265134e5a885e0d46a95c7042 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b7f7fd265134e5a885e0d46a95c7042" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37163 } }
I20260810 11:35:55.886066 29536 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5b7f7fd265134e5a885e0d46a95c7042 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 11:35:55.886248 29536 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5b7f7fd265134e5a885e0d46a95c7042 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 11:35:55.886420 29536 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5b7f7fd265134e5a885e0d46a95c7042 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:35:55.888917 29536 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5b7f7fd265134e5a885e0d46a95c7042 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b7f7fd265134e5a885e0d46a95c7042" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37163 } }
I20260810 11:35:55.889588 29536 leader_election.cc:302] T 00000000000000000000000000000000 P 5b7f7fd265134e5a885e0d46a95c7042 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5b7f7fd265134e5a885e0d46a95c7042; no voters:
I20260810 11:35:55.890398 29536 leader_election.cc:288] T 00000000000000000000000000000000 P 5b7f7fd265134e5a885e0d46a95c7042 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 11:35:55.890494 29546 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5b7f7fd265134e5a885e0d46a95c7042 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:35:55.891364 29546 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5b7f7fd265134e5a885e0d46a95c7042 [term 1 LEADER]: Becoming Leader. State: Replica: 5b7f7fd265134e5a885e0d46a95c7042, State: Running, Role: LEADER
I20260810 11:35:55.891961 29546 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5b7f7fd265134e5a885e0d46a95c7042 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b7f7fd265134e5a885e0d46a95c7042" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37163 } }
I20260810 11:35:55.892831 29536 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5b7f7fd265134e5a885e0d46a95c7042 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 11:35:55.895092 29547 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b7f7fd265134e5a885e0d46a95c7042 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5b7f7fd265134e5a885e0d46a95c7042" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b7f7fd265134e5a885e0d46a95c7042" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37163 } } }
I20260810 11:35:55.895188 29548 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b7f7fd265134e5a885e0d46a95c7042 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5b7f7fd265134e5a885e0d46a95c7042. Latest consensus state: current_term: 1 leader_uuid: "5b7f7fd265134e5a885e0d46a95c7042" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b7f7fd265134e5a885e0d46a95c7042" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37163 } } }
I20260810 11:35:55.895530 29547 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b7f7fd265134e5a885e0d46a95c7042 [sys.catalog]: This master's current role is: LEADER
I20260810 11:35:55.895604 29548 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b7f7fd265134e5a885e0d46a95c7042 [sys.catalog]: This master's current role is: LEADER
I20260810 11:35:55.899986 29552 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 11:35:55.906231 29552 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 11:35:55.916838 29552 catalog_manager.cc:1347] Generated new cluster ID: 59f8c346d34740f090cb20451ff617d4
I20260810 11:35:55.916994 29552 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 11:35:55.943747 29552 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 11:35:55.944309 29552 catalog_manager.cc:1504] Loading token signing keys...
I20260810 11:35:55.951735 29552 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5b7f7fd265134e5a885e0d46a95c7042: Generated new TSK 0
I20260810 11:35:55.953078 29552 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 11:35:55.969844 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 29467
W20260810 11:35:55.971371 29531 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:52495
W20260810 11:35:55.972174 8267 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 29467: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 11:35:55.972347 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 29467
2026-08-10T11:35:55Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (2109 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786361401922362-8267-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 11:35:56 dist-test-slave-srgv krb5kdc[29568](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 11:35:56 dist-test-slave-srgv krb5kdc[29568](info): set up 2 sockets
Aug 10 11:35:56 dist-test-slave-srgv krb5kdc[29568](info): commencing operation
krb5kdc: starting...
W20260810 11:35:57.160946 8267 mini_kdc.cc:121] Time spent starting KDC: real 1.151s user 0.006s sys 0.039s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786361401922362-8267-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786361401922362-8267-0/krb5kdc/test-user.keytab.
Aug 10 11:35:57 dist-test-slave-srgv krb5kdc[29568](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361757, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T11:35:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T11:35:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786361401922362-8267-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786361401922362-8267-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 11:35:57.497838 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786361401922362-8267-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786361401922362-8267-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786361401922362-8267-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44041
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786361401922362-8267-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43487
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44041
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786361401922362-8267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786361401922362-8267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786361401922362-8267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:35:57.875504 29582 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:35:57.875840 29582 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:35:57.875941 29582 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:35:57.883668 29582 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 11:35:57.883816 29582 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:35:57.883867 29582 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:35:57.883934 29582 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 11:35:57.883980 29582 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 11:35:57.895350 29582 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43487
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786361401922362-8267-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786361401922362-8267-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44041
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786361401922362-8267-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44041
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786361401922362-8267-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:35:57.896759 29582 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:35:57.898277 29582 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:35:57.912323 29590 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:35:57.912281 29588 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:35:57.912348 29587 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:35:57.913528 29582 server_base.cc:1034] running on GCE node
I20260810 11:35:57.914494 29582 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:35:57.916168 29582 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:35:57.917569 29582 hybrid_clock.cc:648] HybridClock initialized: now 1786361757917520 us; error 60 us; skew 500 ppm
I20260810 11:35:57.918015 29582 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:35:57 dist-test-slave-srgv krb5kdc[29568](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361757, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:35:57.927882 29582 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 11:35:57.929929 29582 webserver.cc:460] Webserver started at http://127.0.0.1:42383/ using document root <none> and password file <none>
I20260810 11:35:57.930938 29582 fs_manager.cc:359] Metadata directory not provided
I20260810 11:35:57.931123 29582 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:35:57.931519 29582 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:35:57.935480 29582 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786361401922362-8267-0/minicluster-data/master-0/data/instance:
uuid: "ce34fce15426415ca1e75de94e602926"
format_stamp: "Formatted at 2026-08-10 11:35:57 on dist-test-slave-srgv"
server_key: "bfd3e9953fe5b5ce9847a981bb67863e"
server_key_iv: "9aee3f3a2e09e35df092cbd9ec141df5"
server_key_version: "encryptionkey@0"
I20260810 11:35:57.936780 29582 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786361401922362-8267-0/minicluster-data/master-0/wal/instance:
uuid: "ce34fce15426415ca1e75de94e602926"
format_stamp: "Formatted at 2026-08-10 11:35:57 on dist-test-slave-srgv"
server_key: "bfd3e9953fe5b5ce9847a981bb67863e"
server_key_iv: "9aee3f3a2e09e35df092cbd9ec141df5"
server_key_version: "encryptionkey@0"
I20260810 11:35:57.944448 29582 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.009s sys 0.000s
I20260810 11:35:57.949498 29598 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:35:57.952337 29582 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.004s sys 0.000s
I20260810 11:35:57.952596 29582 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786361401922362-8267-0/minicluster-data/master-0/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786361401922362-8267-0/minicluster-data/master-0/wal
uuid: "ce34fce15426415ca1e75de94e602926"
format_stamp: "Formatted at 2026-08-10 11:35:57 on dist-test-slave-srgv"
server_key: "bfd3e9953fe5b5ce9847a981bb67863e"
server_key_iv: "9aee3f3a2e09e35df092cbd9ec141df5"
server_key_version: "encryptionkey@0"
I20260810 11:35:57.952853 29582 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786361401922362-8267-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786361401922362-8267-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786361401922362-8267-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:35:57.976384 29582 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:35:57.981551 29582 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:35:57.981890 29582 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:35:58.003213 29582 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44041
I20260810 11:35:58.003316 29650 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44041 every 8 connection(s)
I20260810 11:35:58.004858 29582 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
I20260810 11:35:58.009135 29651 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:35:58.013036 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 29582
I20260810 11:35:58.013264 8267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786361401922362-8267-0/minicluster-data/master-0/wal/instance
I20260810 11:35:58.013886 8267 external_mini_cluster.cc:1428] Setting key 95f9c3bf15cf9fe4b26d83ab914dac14
I20260810 11:35:58.022585 29651 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce34fce15426415ca1e75de94e602926: Bootstrap starting.
I20260810 11:35:58.026091 29651 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ce34fce15426415ca1e75de94e602926: Neither blocks nor log segments found. Creating new log.
I20260810 11:35:58.027228 29651 log.cc:824] T 00000000000000000000000000000000 P ce34fce15426415ca1e75de94e602926: Log is configured to *not* fsync() on all Append() calls
Aug 10 11:35:58 dist-test-slave-srgv krb5kdc[29568](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361757, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260810 11:35:58.030857 29651 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce34fce15426415ca1e75de94e602926: No bootstrap required, opened a new log
I20260810 11:35:58.041114 29654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:58.016054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60701 (local address 127.0.0.1:44041)
0810 11:35:58.016903 (+ 849us) server_negotiation.cc:206] Beginning negotiation
0810 11:35:58.016916 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 11:35:58.016961 (+ 45us) server_negotiation.cc:407] Connection header received
0810 11:35:58.018963 (+ 2002us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:35:58.018990 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:35:58.019443 (+ 453us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:35:58.019906 (+ 463us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:35:58.021460 (+ 1554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:58.022919 (+ 1459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:58.024294 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:58.024988 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:58.025197 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:35:58.030348 (+ 5151us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:35:58.030362 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:35:58.030382 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:35:58.030442 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:35:58.035148 (+ 4706us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:35:58.036165 (+ 1017us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:35:58.036177 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:35:58.036187 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:35:58.036468 (+ 281us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:35:58.037027 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:35:58.037033 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:35:58.037037 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:35:58.037982 (+ 945us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:35:58.038337 (+ 355us) server_negotiation.cc:1035] Waiting for connection context
0810 11:35:58.038879 (+ 542us) server_negotiation.cc:299] Negotiation successful
0810 11:35:58.039450 (+ 571us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":627,"thread_start_us":377,"threads_started":1}
I20260810 11:35:58.042560 29651 raft_consensus.cc:348] T 00000000000000000000000000000000 P ce34fce15426415ca1e75de94e602926 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce34fce15426415ca1e75de94e602926" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44041 } }
I20260810 11:35:58.043140 29651 raft_consensus.cc:374] T 00000000000000000000000000000000 P ce34fce15426415ca1e75de94e602926 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:35:58.043280 29651 raft_consensus.cc:729] T 00000000000000000000000000000000 P ce34fce15426415ca1e75de94e602926 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce34fce15426415ca1e75de94e602926, State: Initialized, Role: FOLLOWER
I20260810 11:35:58.044055 29651 consensus_queue.cc:260] T 00000000000000000000000000000000 P ce34fce15426415ca1e75de94e602926 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce34fce15426415ca1e75de94e602926" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44041 } }
I20260810 11:35:58.044381 29651 raft_consensus.cc:388] T 00000000000000000000000000000000 P ce34fce15426415ca1e75de94e602926 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 11:35:58.044554 29651 raft_consensus.cc:482] T 00000000000000000000000000000000 P ce34fce15426415ca1e75de94e602926 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 11:35:58.044759 29651 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ce34fce15426415ca1e75de94e602926 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:35:58.046567 8267 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 11:35:58.047874 29651 raft_consensus.cc:504] T 00000000000000000000000000000000 P ce34fce15426415ca1e75de94e602926 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce34fce15426415ca1e75de94e602926" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44041 } }
I20260810 11:35:58.048476 29651 leader_election.cc:302] T 00000000000000000000000000000000 P ce34fce15426415ca1e75de94e602926 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ce34fce15426415ca1e75de94e602926; no voters:
I20260810 11:35:58.049150 29651 leader_election.cc:288] T 00000000000000000000000000000000 P ce34fce15426415ca1e75de94e602926 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 11:35:58.049450 29656 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ce34fce15426415ca1e75de94e602926 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:35:58.051498 29651 sys_catalog.cc:564] T 00000000000000000000000000000000 P ce34fce15426415ca1e75de94e602926 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 11:35:58.053805 29656 raft_consensus.cc:686] T 00000000000000000000000000000000 P ce34fce15426415ca1e75de94e602926 [term 1 LEADER]: Becoming Leader. State: Replica: ce34fce15426415ca1e75de94e602926, State: Running, Role: LEADER
I20260810 11:35:58.054940 29656 consensus_queue.cc:237] T 00000000000000000000000000000000 P ce34fce15426415ca1e75de94e602926 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce34fce15426415ca1e75de94e602926" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44041 } }
I20260810 11:35:58.059255 29659 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce34fce15426415ca1e75de94e602926 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ce34fce15426415ca1e75de94e602926. Latest consensus state: current_term: 1 leader_uuid: "ce34fce15426415ca1e75de94e602926" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce34fce15426415ca1e75de94e602926" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44041 } } }
I20260810 11:35:58.059810 29659 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce34fce15426415ca1e75de94e602926 [sys.catalog]: This master's current role is: LEADER
I20260810 11:35:58.060570 29658 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce34fce15426415ca1e75de94e602926 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ce34fce15426415ca1e75de94e602926" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce34fce15426415ca1e75de94e602926" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44041 } } }
I20260810 11:35:58.061007 29658 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce34fce15426415ca1e75de94e602926 [sys.catalog]: This master's current role is: LEADER
I20260810 11:35:58.081713 29666 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 11:35:58.099071 29666 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 11:35:58.116245 29654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:35:58.078695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56639 (local address 127.0.0.1:44041)
0810 11:35:58.079001 (+ 306us) server_negotiation.cc:206] Beginning negotiation
0810 11:35:58.079012 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 11:35:58.090477 (+ 11465us) server_negotiation.cc:407] Connection header received
0810 11:35:58.091050 (+ 573us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:35:58.091057 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:35:58.091146 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:35:58.091224 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:35:58.093726 (+ 2502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:58.095052 (+ 1326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:58.096961 (+ 1909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:35:58.097727 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:35:58.097854 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:35:58.101314 (+ 3460us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:35:58.101322 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:35:58.101327 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:35:58.101375 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:35:58.112537 (+ 11162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:35:58.113722 (+ 1185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:35:58.113737 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:35:58.113739 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:35:58.113916 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:35:58.114404 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:35:58.114410 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:35:58.114413 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:35:58.114814 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:35:58.115125 (+ 311us) server_negotiation.cc:1035] Waiting for connection context
0810 11:35:58.115570 (+ 445us) server_negotiation.cc:299] Negotiation successful
0810 11:35:58.115849 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111}
I20260810 11:35:58.119310 29666 catalog_manager.cc:1347] Generated new cluster ID: 4a782b1421f64135abdc1edfac8d78b6
I20260810 11:35:58.119448 29666 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 11:35:58.127411 29666 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 11:35:58.129303 29666 catalog_manager.cc:1504] Loading token signing keys...
I20260810 11:35:58.138803 29666 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ce34fce15426415ca1e75de94e602926: Generated new TSK 0
I20260810 11:35:58.139869 29666 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 11:35:58.150540 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 29582
I20260810 11:35:58.151767 29646 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:35:58.245951 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 29582
2026-08-10T11:35:58Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (2276 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786361401922362-8267-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 11:35:58 dist-test-slave-srgv krb5kdc[29684](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 11:35:58 dist-test-slave-srgv krb5kdc[29684](info): set up 2 sockets
Aug 10 11:35:58 dist-test-slave-srgv krb5kdc[29684](info): commencing operation
krb5kdc: starting...
W20260810 11:35:59.432942 8267 mini_kdc.cc:121] Time spent starting KDC: real 1.147s user 0.000s sys 0.052s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786361401922362-8267-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786361401922362-8267-0/krb5kdc/test-user.keytab.
Aug 10 11:35:59 dist-test-slave-srgv krb5kdc[29684](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361759, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T11:35:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T11:35:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786361401922362-8267-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786361401922362-8267-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 11:35:59.775317 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786361401922362-8267-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786361401922362-8267-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786361401922362-8267-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41155
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786361401922362-8267-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41155
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786361401922362-8267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786361401922362-8267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786361401922362-8267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:36:00.159160 29697 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:36:00.159531 29697 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:36:00.159772 29697 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:36:00.167635 29697 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 11:36:00.167788 29697 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:36:00.167894 29697 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:36:00.168023 29697 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 11:36:00.168108 29697 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 11:36:00.178840 29697 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786361401922362-8267-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786361401922362-8267-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41155
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786361401922362-8267-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41155
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786361401922362-8267-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:36:00.180455 29697 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:36:00.182128 29697 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:36:00.196924 29705 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:36:00.197105 29702 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:36:00.197711 29703 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:36:00.197887 29697 server_base.cc:1034] running on GCE node
I20260810 11:36:00.198890 29697 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:36:00.200595 29697 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:36:00.201889 29697 hybrid_clock.cc:648] HybridClock initialized: now 1786361760201838 us; error 49 us; skew 500 ppm
I20260810 11:36:00.202351 29697 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:36:00 dist-test-slave-srgv krb5kdc[29684](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361760, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:36:00.206220 29697 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 11:36:00.208135 29697 webserver.cc:460] Webserver started at http://127.0.0.1:42629/ using document root <none> and password file <none>
I20260810 11:36:00.208995 29697 fs_manager.cc:359] Metadata directory not provided
I20260810 11:36:00.209182 29697 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:36:00.209633 29697 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:36:00.213270 29697 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786361401922362-8267-0/minicluster-data/master-0/data/instance:
uuid: "e44de729915947828276e50b29092c37"
format_stamp: "Formatted at 2026-08-10 11:36:00 on dist-test-slave-srgv"
server_key: "1ec983e19050985582da00776878c7a4"
server_key_iv: "80b389780eff2acbe6cbed012654f18a"
server_key_version: "encryptionkey@0"
I20260810 11:36:00.214349 29697 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786361401922362-8267-0/minicluster-data/master-0/wal/instance:
uuid: "e44de729915947828276e50b29092c37"
format_stamp: "Formatted at 2026-08-10 11:36:00 on dist-test-slave-srgv"
server_key: "1ec983e19050985582da00776878c7a4"
server_key_iv: "80b389780eff2acbe6cbed012654f18a"
server_key_version: "encryptionkey@0"
I20260810 11:36:00.221001 29697 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260810 11:36:00.225312 29713 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:36:00.227123 29697 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260810 11:36:00.227411 29697 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786361401922362-8267-0/minicluster-data/master-0/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786361401922362-8267-0/minicluster-data/master-0/wal
uuid: "e44de729915947828276e50b29092c37"
format_stamp: "Formatted at 2026-08-10 11:36:00 on dist-test-slave-srgv"
server_key: "1ec983e19050985582da00776878c7a4"
server_key_iv: "80b389780eff2acbe6cbed012654f18a"
server_key_version: "encryptionkey@0"
I20260810 11:36:00.227671 29697 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786361401922362-8267-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786361401922362-8267-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786361401922362-8267-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:36:00.249976 29697 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:36:00.255409 29697 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:36:00.255801 29697 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:36:00.278060 29697 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41155
I20260810 11:36:00.278127 29766 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41155 every 8 connection(s)
I20260810 11:36:00.279650 29697 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
I20260810 11:36:00.281098 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 29697
I20260810 11:36:00.281327 8267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786361401922362-8267-0/minicluster-data/master-0/wal/instance
I20260810 11:36:00.281950 8267 external_mini_cluster.cc:1428] Setting key 34e3a9cbba7ab27fa8f02a5d4252ed8e
I20260810 11:36:00.285372 29767 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:36:00.297686 29770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:36:00.284140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59243 (local address 127.0.0.1:41155)
0810 11:36:00.285145 (+ 1005us) server_negotiation.cc:206] Beginning negotiation
0810 11:36:00.285159 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 11:36:00.285220 (+ 61us) server_negotiation.cc:407] Connection header received
0810 11:36:00.287259 (+ 2039us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:36:00.287288 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:36:00.287761 (+ 473us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:36:00.288274 (+ 513us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:36:00.288960 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:36:00.290532 (+ 1572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:36:00.292090 (+ 1558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:36:00.292887 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:36:00.293112 (+ 225us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:36:00.294754 (+ 1642us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:36:00.294763 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:36:00.294786 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:36:00.295601 (+ 815us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:36:00.295706 (+ 105us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:36:00.295738 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:36:00.295899 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0810 11:36:00.296160 (+ 261us) server_negotiation.cc:299] Negotiation successful
0810 11:36:00.296483 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":758,"thread_start_us":477,"threads_started":1}
I20260810 11:36:00.298409 29767 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e44de729915947828276e50b29092c37: Bootstrap starting.
I20260810 11:36:00.301855 8267 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 11:36:00.302211 29767 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e44de729915947828276e50b29092c37: Neither blocks nor log segments found. Creating new log.
I20260810 11:36:00.303350 29767 log.cc:824] T 00000000000000000000000000000000 P e44de729915947828276e50b29092c37: Log is configured to *not* fsync() on all Append() calls
I20260810 11:36:00.306886 29767 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e44de729915947828276e50b29092c37: No bootstrap required, opened a new log
I20260810 11:36:00.313720 29770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:36:00.306897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42609 (local address 127.0.0.1:41155)
0810 11:36:00.307136 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0810 11:36:00.307146 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 11:36:00.307703 (+ 557us) server_negotiation.cc:407] Connection header received
0810 11:36:00.307943 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:36:00.307947 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:36:00.308010 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:36:00.308130 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:36:00.308688 (+ 558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:36:00.309560 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:36:00.310996 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:36:00.311489 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:36:00.311644 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:36:00.312902 (+ 1258us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:36:00.312907 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:36:00.312911 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:36:00.312930 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:36:00.312977 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:36:00.312994 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:36:00.313105 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0810 11:36:00.313262 (+ 157us) server_negotiation.cc:299] Negotiation successful
0810 11:36:00.313363 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260810 11:36:00.316102 29767 raft_consensus.cc:348] T 00000000000000000000000000000000 P e44de729915947828276e50b29092c37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e44de729915947828276e50b29092c37" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41155 } }
I20260810 11:36:00.316589 29767 raft_consensus.cc:374] T 00000000000000000000000000000000 P e44de729915947828276e50b29092c37 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:36:00.316728 29767 raft_consensus.cc:729] T 00000000000000000000000000000000 P e44de729915947828276e50b29092c37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e44de729915947828276e50b29092c37, State: Initialized, Role: FOLLOWER
I20260810 11:36:00.317333 29767 consensus_queue.cc:260] T 00000000000000000000000000000000 P e44de729915947828276e50b29092c37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e44de729915947828276e50b29092c37" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41155 } }
I20260810 11:36:00.317703 29767 raft_consensus.cc:388] T 00000000000000000000000000000000 P e44de729915947828276e50b29092c37 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 11:36:00.317893 29767 raft_consensus.cc:482] T 00000000000000000000000000000000 P e44de729915947828276e50b29092c37 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 11:36:00.318096 29767 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e44de729915947828276e50b29092c37 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:36:00.320498 29767 raft_consensus.cc:504] T 00000000000000000000000000000000 P e44de729915947828276e50b29092c37 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e44de729915947828276e50b29092c37" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41155 } }
I20260810 11:36:00.321089 29767 leader_election.cc:302] T 00000000000000000000000000000000 P e44de729915947828276e50b29092c37 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e44de729915947828276e50b29092c37; no voters:
I20260810 11:36:00.321796 29767 leader_election.cc:288] T 00000000000000000000000000000000 P e44de729915947828276e50b29092c37 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 11:36:00.321946 29777 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e44de729915947828276e50b29092c37 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:36:00.322708 29777 raft_consensus.cc:686] T 00000000000000000000000000000000 P e44de729915947828276e50b29092c37 [term 1 LEADER]: Becoming Leader. State: Replica: e44de729915947828276e50b29092c37, State: Running, Role: LEADER
I20260810 11:36:00.323401 29777 consensus_queue.cc:237] T 00000000000000000000000000000000 P e44de729915947828276e50b29092c37 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e44de729915947828276e50b29092c37" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41155 } }
I20260810 11:36:00.324152 29767 sys_catalog.cc:564] T 00000000000000000000000000000000 P e44de729915947828276e50b29092c37 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 11:36:00.326491 29779 sys_catalog.cc:455] T 00000000000000000000000000000000 P e44de729915947828276e50b29092c37 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e44de729915947828276e50b29092c37. Latest consensus state: current_term: 1 leader_uuid: "e44de729915947828276e50b29092c37" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e44de729915947828276e50b29092c37" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41155 } } }
I20260810 11:36:00.327028 29779 sys_catalog.cc:458] T 00000000000000000000000000000000 P e44de729915947828276e50b29092c37 [sys.catalog]: This master's current role is: LEADER
I20260810 11:36:00.327698 29778 sys_catalog.cc:455] T 00000000000000000000000000000000 P e44de729915947828276e50b29092c37 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e44de729915947828276e50b29092c37" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e44de729915947828276e50b29092c37" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41155 } } }
I20260810 11:36:00.328061 29778 sys_catalog.cc:458] T 00000000000000000000000000000000 P e44de729915947828276e50b29092c37 [sys.catalog]: This master's current role is: LEADER
I20260810 11:36:00.334033 29783 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 11:36:00.340157 29783 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 11:36:00.351989 29783 catalog_manager.cc:1347] Generated new cluster ID: ec4577cdba8f49e0a7b4662fb848095d
I20260810 11:36:00.352138 29783 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 11:36:00.366297 29783 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 11:36:00.367877 29783 catalog_manager.cc:1504] Loading token signing keys...
I20260810 11:36:00.383460 29783 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e44de729915947828276e50b29092c37: Generated new TSK 0
I20260810 11:36:00.384608 29783 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 11:36:00.395716 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 29697
W20260810 11:36:00.397028 29762 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:59243
W20260810 11:36:00.397953 8267 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 29697: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 11:36:00.398149 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 29697
2026-08-10T11:36:00Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (2152 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786361401922362-8267-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 11:36:00 dist-test-slave-srgv krb5kdc[29799](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 11:36:00 dist-test-slave-srgv krb5kdc[29799](info): set up 2 sockets
Aug 10 11:36:00 dist-test-slave-srgv krb5kdc[29799](info): commencing operation
krb5kdc: starting...
W20260810 11:36:01.569375 8267 mini_kdc.cc:121] Time spent starting KDC: real 1.131s user 0.008s sys 0.038s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786361401922362-8267-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786361401922362-8267-0/krb5kdc/test-user.keytab.
Aug 10 11:36:01 dist-test-slave-srgv krb5kdc[29799](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361761, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T11:36:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T11:36:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786361401922362-8267-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786361401922362-8267-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 11:36:01.910334 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786361401922362-8267-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786361401922362-8267-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786361401922362-8267-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43519
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786361401922362-8267-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44607
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43519
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786361401922362-8267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786361401922362-8267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786361401922362-8267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:36:02.285679 29812 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:36:02.286033 29812 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:36:02.286221 29812 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:36:02.293931 29812 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 11:36:02.294083 29812 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:36:02.294216 29812 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:36:02.294345 29812 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 11:36:02.294440 29812 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 11:36:02.304965 29812 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44607
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786361401922362-8267-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786361401922362-8267-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43519
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786361401922362-8267-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43519
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786361401922362-8267-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:36:02.306531 29812 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:36:02.307967 29812 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:36:02.321815 29818 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:36:02.321893 29820 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:36:02.321795 29817 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:36:02.323262 29812 server_base.cc:1034] running on GCE node
I20260810 11:36:02.324162 29812 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:36:02.325750 29812 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:36:02.327006 29812 hybrid_clock.cc:648] HybridClock initialized: now 1786361762326970 us; error 31 us; skew 500 ppm
I20260810 11:36:02.327473 29812 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:36:02 dist-test-slave-srgv krb5kdc[29799](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361762, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:36:02.331442 29812 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 11:36:02.333935 29812 webserver.cc:460] Webserver started at http://127.0.0.1:38731/ using document root <none> and password file <none>
I20260810 11:36:02.334707 29812 fs_manager.cc:359] Metadata directory not provided
I20260810 11:36:02.334870 29812 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:36:02.335217 29812 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:36:02.338550 29812 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786361401922362-8267-0/minicluster-data/master-0/data/instance:
uuid: "4e54204f619a4701b222e330f806ff19"
format_stamp: "Formatted at 2026-08-10 11:36:02 on dist-test-slave-srgv"
server_key: "6f41f81e03aca287326d3d2d5f2da9fc"
server_key_iv: "d2887768b1798dc75754abdfa9ee5de0"
server_key_version: "encryptionkey@0"
I20260810 11:36:02.339411 29812 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786361401922362-8267-0/minicluster-data/master-0/wal/instance:
uuid: "4e54204f619a4701b222e330f806ff19"
format_stamp: "Formatted at 2026-08-10 11:36:02 on dist-test-slave-srgv"
server_key: "6f41f81e03aca287326d3d2d5f2da9fc"
server_key_iv: "d2887768b1798dc75754abdfa9ee5de0"
server_key_version: "encryptionkey@0"
I20260810 11:36:02.345333 29812 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.003s
I20260810 11:36:02.349834 29827 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:36:02.351505 29812 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260810 11:36:02.351784 29812 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786361401922362-8267-0/minicluster-data/master-0/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786361401922362-8267-0/minicluster-data/master-0/wal
uuid: "4e54204f619a4701b222e330f806ff19"
format_stamp: "Formatted at 2026-08-10 11:36:02 on dist-test-slave-srgv"
server_key: "6f41f81e03aca287326d3d2d5f2da9fc"
server_key_iv: "d2887768b1798dc75754abdfa9ee5de0"
server_key_version: "encryptionkey@0"
I20260810 11:36:02.352061 29812 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786361401922362-8267-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786361401922362-8267-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786361401922362-8267-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:36:02.358968 29812 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:36:02.363803 29812 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:36:02.364176 29812 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:36:02.385725 29812 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43519
I20260810 11:36:02.385823 29880 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43519 every 8 connection(s)
I20260810 11:36:02.387360 29812 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
I20260810 11:36:02.389089 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 29812
I20260810 11:36:02.389298 8267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786361401922362-8267-0/minicluster-data/master-0/wal/instance
I20260810 11:36:02.389899 8267 external_mini_cluster.cc:1428] Setting key 456bd234298688ad18471707750783d6
I20260810 11:36:02.392753 29882 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:36:02.398620 29885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:36:02.391933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45733 (local address 127.0.0.1:43519)
0810 11:36:02.392787 (+ 854us) server_negotiation.cc:206] Beginning negotiation
0810 11:36:02.392799 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 11:36:02.392884 (+ 85us) server_negotiation.cc:407] Connection header received
0810 11:36:02.394736 (+ 1852us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:36:02.394760 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:36:02.395132 (+ 372us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:36:02.395589 (+ 457us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:36:02.396769 (+ 1180us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:36:02.396776 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:36:02.396789 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:36:02.396804 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:36:02.396886 (+ 82us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:36:02.396910 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:36:02.397210 (+ 300us) server_negotiation.cc:1035] Waiting for connection context
0810 11:36:02.397359 (+ 149us) server_negotiation.cc:299] Negotiation successful
0810 11:36:02.397596 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":597,"thread_start_us":368,"threads_started":1}
I20260810 11:36:02.403146 8267 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 11:36:02.405962 29882 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e54204f619a4701b222e330f806ff19: Bootstrap starting.
I20260810 11:36:02.410012 29882 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4e54204f619a4701b222e330f806ff19: Neither blocks nor log segments found. Creating new log.
I20260810 11:36:02.411168 29885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:36:02.407950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40725 (local address 127.0.0.1:43519)
0810 11:36:02.408173 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0810 11:36:02.408180 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:36:02.408749 (+ 569us) server_negotiation.cc:407] Connection header received
0810 11:36:02.408910 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:36:02.408915 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:36:02.408995 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:36:02.409123 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:36:02.410442 (+ 1319us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:36:02.410449 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:36:02.410456 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:36:02.410480 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:36:02.410558 (+ 78us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:36:02.410587 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:36:02.410724 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0810 11:36:02.410822 (+ 98us) server_negotiation.cc:299] Negotiation successful
0810 11:36:02.410882 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260810 11:36:02.411573 29882 log.cc:824] T 00000000000000000000000000000000 P 4e54204f619a4701b222e330f806ff19: Log is configured to *not* fsync() on all Append() calls
I20260810 11:36:02.414805 29882 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e54204f619a4701b222e330f806ff19: No bootstrap required, opened a new log
I20260810 11:36:02.423230 29882 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4e54204f619a4701b222e330f806ff19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e54204f619a4701b222e330f806ff19" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43519 } }
I20260810 11:36:02.423804 29882 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4e54204f619a4701b222e330f806ff19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:36:02.423942 29882 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4e54204f619a4701b222e330f806ff19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e54204f619a4701b222e330f806ff19, State: Initialized, Role: FOLLOWER
I20260810 11:36:02.424662 29882 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4e54204f619a4701b222e330f806ff19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e54204f619a4701b222e330f806ff19" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43519 } }
I20260810 11:36:02.424921 29882 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4e54204f619a4701b222e330f806ff19 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 11:36:02.425098 29882 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4e54204f619a4701b222e330f806ff19 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 11:36:02.425285 29882 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4e54204f619a4701b222e330f806ff19 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:36:02.427649 29882 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4e54204f619a4701b222e330f806ff19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e54204f619a4701b222e330f806ff19" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43519 } }
I20260810 11:36:02.428277 29882 leader_election.cc:302] T 00000000000000000000000000000000 P 4e54204f619a4701b222e330f806ff19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4e54204f619a4701b222e330f806ff19; no voters:
I20260810 11:36:02.428925 29882 leader_election.cc:288] T 00000000000000000000000000000000 P 4e54204f619a4701b222e330f806ff19 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 11:36:02.429108 29892 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4e54204f619a4701b222e330f806ff19 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:36:02.429943 29892 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4e54204f619a4701b222e330f806ff19 [term 1 LEADER]: Becoming Leader. State: Replica: 4e54204f619a4701b222e330f806ff19, State: Running, Role: LEADER
I20260810 11:36:02.430596 29892 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4e54204f619a4701b222e330f806ff19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e54204f619a4701b222e330f806ff19" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43519 } }
I20260810 11:36:02.431316 29882 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4e54204f619a4701b222e330f806ff19 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 11:36:02.433908 29894 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e54204f619a4701b222e330f806ff19 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4e54204f619a4701b222e330f806ff19. Latest consensus state: current_term: 1 leader_uuid: "4e54204f619a4701b222e330f806ff19" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e54204f619a4701b222e330f806ff19" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43519 } } }
I20260810 11:36:02.434417 29893 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e54204f619a4701b222e330f806ff19 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4e54204f619a4701b222e330f806ff19" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e54204f619a4701b222e330f806ff19" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43519 } } }
I20260810 11:36:02.434701 29894 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e54204f619a4701b222e330f806ff19 [sys.catalog]: This master's current role is: LEADER
I20260810 11:36:02.434758 29893 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e54204f619a4701b222e330f806ff19 [sys.catalog]: This master's current role is: LEADER
I20260810 11:36:02.436852 29902 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 11:36:02.442970 29902 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 11:36:02.454563 29902 catalog_manager.cc:1347] Generated new cluster ID: 5eb6923cf2b141ff88cf7064108f3510
I20260810 11:36:02.454748 29902 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 11:36:02.471197 29902 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 11:36:02.471730 29902 catalog_manager.cc:1504] Loading token signing keys...
I20260810 11:36:02.490980 29902 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4e54204f619a4701b222e330f806ff19: Generated new TSK 0
I20260810 11:36:02.492030 29902 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 11:36:02.512822 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 29812
W20260810 11:36:02.514359 29876 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:45733
W20260810 11:36:02.515319 8267 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 29812: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 11:36:02.515473 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 29812
2026-08-10T11:36:02Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (2118 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786361401922362-8267-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 11:36:02 dist-test-slave-srgv krb5kdc[29914](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 11:36:02 dist-test-slave-srgv krb5kdc[29914](info): set up 2 sockets
Aug 10 11:36:02 dist-test-slave-srgv krb5kdc[29914](info): commencing operation
krb5kdc: starting...
W20260810 11:36:03.668879 8267 mini_kdc.cc:121] Time spent starting KDC: real 1.111s user 0.003s sys 0.038s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786361401922362-8267-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786361401922362-8267-0/krb5kdc/test-user.keytab.
Aug 10 11:36:03 dist-test-slave-srgv krb5kdc[29914](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361763, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T11:36:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T11:36:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786361401922362-8267-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786361401922362-8267-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 11:36:04.009927 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786361401922362-8267-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786361401922362-8267-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786361401922362-8267-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39259
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786361401922362-8267-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34427
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39259
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786361401922362-8267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786361401922362-8267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786361401922362-8267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:36:04.370080 29928 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:36:04.370507 29928 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:36:04.370702 29928 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:36:04.378388 29928 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 11:36:04.378598 29928 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:36:04.378721 29928 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:36:04.378837 29928 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 11:36:04.378927 29928 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 11:36:04.390460 29928 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34427
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786361401922362-8267-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786361401922362-8267-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39259
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786361401922362-8267-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39259
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786361401922362-8267-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:36:04.391950 29928 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:36:04.393532 29928 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:36:04.406625 29934 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:36:04.406792 29933 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:36:04.406792 29936 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:36:04.408074 29928 server_base.cc:1034] running on GCE node
I20260810 11:36:04.408941 29928 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:36:04.410634 29928 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:36:04.411923 29928 hybrid_clock.cc:648] HybridClock initialized: now 1786361764411851 us; error 59 us; skew 500 ppm
I20260810 11:36:04.412290 29928 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:36:04 dist-test-slave-srgv krb5kdc[29914](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361764, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:36:04.422262 29928 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 11:36:04.424134 29928 webserver.cc:460] Webserver started at http://127.0.0.1:43137/ using document root <none> and password file <none>
I20260810 11:36:04.424950 29928 fs_manager.cc:359] Metadata directory not provided
I20260810 11:36:04.425094 29928 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:36:04.425494 29928 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:36:04.428751 29928 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786361401922362-8267-0/minicluster-data/master-0/data/instance:
uuid: "3449b0cbf10e49c8a9dcc0621e3b0011"
format_stamp: "Formatted at 2026-08-10 11:36:04 on dist-test-slave-srgv"
server_key: "fc875fd1355c6545eec6dff068a20bbc"
server_key_iv: "deaa8bbc46954aad8595d0b1d9841b72"
server_key_version: "encryptionkey@0"
I20260810 11:36:04.429674 29928 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786361401922362-8267-0/minicluster-data/master-0/wal/instance:
uuid: "3449b0cbf10e49c8a9dcc0621e3b0011"
format_stamp: "Formatted at 2026-08-10 11:36:04 on dist-test-slave-srgv"
server_key: "fc875fd1355c6545eec6dff068a20bbc"
server_key_iv: "deaa8bbc46954aad8595d0b1d9841b72"
server_key_version: "encryptionkey@0"
I20260810 11:36:04.436070 29928 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260810 11:36:04.440126 29943 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:36:04.441723 29928 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260810 11:36:04.441941 29928 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786361401922362-8267-0/minicluster-data/master-0/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786361401922362-8267-0/minicluster-data/master-0/wal
uuid: "3449b0cbf10e49c8a9dcc0621e3b0011"
format_stamp: "Formatted at 2026-08-10 11:36:04 on dist-test-slave-srgv"
server_key: "fc875fd1355c6545eec6dff068a20bbc"
server_key_iv: "deaa8bbc46954aad8595d0b1d9841b72"
server_key_version: "encryptionkey@0"
I20260810 11:36:04.442152 29928 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786361401922362-8267-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786361401922362-8267-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786361401922362-8267-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:36:04.458745 29928 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:36:04.463068 29928 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:36:04.463399 29928 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:36:04.483973 29928 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39259
I20260810 11:36:04.484037 29995 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39259 every 8 connection(s)
I20260810 11:36:04.485589 29928 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
I20260810 11:36:04.489697 29996 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:36:04.493978 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 29928
I20260810 11:36:04.494217 8267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786361401922362-8267-0/minicluster-data/master-0/wal/instance
I20260810 11:36:04.494868 8267 external_mini_cluster.cc:1428] Setting key d6ad75fb1f764f6fc4ecf5da42882196
I20260810 11:36:04.503892 29996 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3449b0cbf10e49c8a9dcc0621e3b0011: Bootstrap starting.
I20260810 11:36:04.507560 29996 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3449b0cbf10e49c8a9dcc0621e3b0011: Neither blocks nor log segments found. Creating new log.
I20260810 11:36:04.508747 29996 log.cc:824] T 00000000000000000000000000000000 P 3449b0cbf10e49c8a9dcc0621e3b0011: Log is configured to *not* fsync() on all Append() calls
Aug 10 11:36:04 dist-test-slave-srgv krb5kdc[29914](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361763, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260810 11:36:04.512437 29996 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3449b0cbf10e49c8a9dcc0621e3b0011: No bootstrap required, opened a new log
I20260810 11:36:04.520603 29999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:36:04.497139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.154:42579 (local address 127.0.0.1:39259)
0810 11:36:04.498120 (+ 981us) server_negotiation.cc:206] Beginning negotiation
0810 11:36:04.498138 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0810 11:36:04.498199 (+ 61us) server_negotiation.cc:407] Connection header received
0810 11:36:04.500086 (+ 1887us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:36:04.500122 (+ 36us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:36:04.500623 (+ 501us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:36:04.501102 (+ 479us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:36:04.502766 (+ 1664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:36:04.504253 (+ 1487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:36:04.505744 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:36:04.506463 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:36:04.506685 (+ 222us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:36:04.511907 (+ 5222us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:36:04.511916 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:36:04.511932 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:36:04.511969 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:36:04.515501 (+ 3532us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:36:04.516643 (+ 1142us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:36:04.516650 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:36:04.516657 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:36:04.516879 (+ 222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:36:04.517463 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:36:04.517469 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:36:04.517472 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:36:04.518111 (+ 639us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:36:04.518371 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0810 11:36:04.518928 (+ 557us) server_negotiation.cc:299] Negotiation successful
0810 11:36:04.519356 (+ 428us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":723,"thread_start_us":434,"threads_started":1}
I20260810 11:36:04.522569 29996 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3449b0cbf10e49c8a9dcc0621e3b0011 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3449b0cbf10e49c8a9dcc0621e3b0011" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39259 } }
I20260810 11:36:04.523219 29996 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3449b0cbf10e49c8a9dcc0621e3b0011 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:36:04.523368 29996 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3449b0cbf10e49c8a9dcc0621e3b0011 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3449b0cbf10e49c8a9dcc0621e3b0011, State: Initialized, Role: FOLLOWER
I20260810 11:36:04.524160 29996 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3449b0cbf10e49c8a9dcc0621e3b0011 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3449b0cbf10e49c8a9dcc0621e3b0011" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39259 } }
I20260810 11:36:04.524508 29996 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3449b0cbf10e49c8a9dcc0621e3b0011 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 11:36:04.524668 29996 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3449b0cbf10e49c8a9dcc0621e3b0011 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 11:36:04.524863 29996 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3449b0cbf10e49c8a9dcc0621e3b0011 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:36:04.526320 8267 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 11:36:04.528434 29996 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3449b0cbf10e49c8a9dcc0621e3b0011 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3449b0cbf10e49c8a9dcc0621e3b0011" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39259 } }
I20260810 11:36:04.529152 29996 leader_election.cc:302] T 00000000000000000000000000000000 P 3449b0cbf10e49c8a9dcc0621e3b0011 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3449b0cbf10e49c8a9dcc0621e3b0011; no voters:
I20260810 11:36:04.529963 29996 leader_election.cc:288] T 00000000000000000000000000000000 P 3449b0cbf10e49c8a9dcc0621e3b0011 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 11:36:04.530128 30001 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3449b0cbf10e49c8a9dcc0621e3b0011 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:36:04.531752 30001 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3449b0cbf10e49c8a9dcc0621e3b0011 [term 1 LEADER]: Becoming Leader. State: Replica: 3449b0cbf10e49c8a9dcc0621e3b0011, State: Running, Role: LEADER
I20260810 11:36:04.532421 30001 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3449b0cbf10e49c8a9dcc0621e3b0011 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3449b0cbf10e49c8a9dcc0621e3b0011" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39259 } }
I20260810 11:36:04.533010 29996 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3449b0cbf10e49c8a9dcc0621e3b0011 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 11:36:04.542119 30004 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3449b0cbf10e49c8a9dcc0621e3b0011 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3449b0cbf10e49c8a9dcc0621e3b0011" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3449b0cbf10e49c8a9dcc0621e3b0011" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39259 } } }
I20260810 11:36:04.542757 30004 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3449b0cbf10e49c8a9dcc0621e3b0011 [sys.catalog]: This master's current role is: LEADER
I20260810 11:36:04.543588 30005 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3449b0cbf10e49c8a9dcc0621e3b0011 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3449b0cbf10e49c8a9dcc0621e3b0011. Latest consensus state: current_term: 1 leader_uuid: "3449b0cbf10e49c8a9dcc0621e3b0011" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3449b0cbf10e49c8a9dcc0621e3b0011" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39259 } } }
I20260810 11:36:04.544293 30005 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3449b0cbf10e49c8a9dcc0621e3b0011 [sys.catalog]: This master's current role is: LEADER
I20260810 11:36:04.554284 30014 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 11:36:04.560896 30014 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 11:36:04.562354 30023 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260810 11:36:04.565210 29999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:36:04.539707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.154:47817 (local address 127.0.0.1:39259)
0810 11:36:04.539941 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0810 11:36:04.539951 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 11:36:04.540799 (+ 848us) server_negotiation.cc:407] Connection header received
0810 11:36:04.541177 (+ 378us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:36:04.541183 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:36:04.541270 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:36:04.541356 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:36:04.544065 (+ 2709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:36:04.545763 (+ 1698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:36:04.548352 (+ 2589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:36:04.549349 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:36:04.549569 (+ 220us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:36:04.555023 (+ 5454us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:36:04.555032 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:36:04.555037 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:36:04.555082 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:36:04.560316 (+ 5234us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:36:04.562018 (+ 1702us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:36:04.562025 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:36:04.562028 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:36:04.562215 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:36:04.562948 (+ 733us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:36:04.562959 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:36:04.562964 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:36:04.563558 (+ 594us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:36:04.563864 (+ 306us) server_negotiation.cc:1035] Waiting for connection context
0810 11:36:04.564476 (+ 612us) server_negotiation.cc:299] Negotiation successful
0810 11:36:04.564722 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260810 11:36:04.571872 30014 catalog_manager.cc:1347] Generated new cluster ID: 2741bc27005647a7bb903cd2f4f907e3
I20260810 11:36:04.572052 30014 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 11:36:04.586175 30014 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 11:36:04.587639 30014 catalog_manager.cc:1504] Loading token signing keys...
I20260810 11:36:04.603662 30014 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3449b0cbf10e49c8a9dcc0621e3b0011: Generated new TSK 0
I20260810 11:36:04.604725 30014 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 11:36:04.615480 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 29928
I20260810 11:36:04.616858 29991 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:36:04.712822 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 29928
W20260810 11:36:04.732880 29912 connection.cc:535] client connection to 127.0.0.1:39259 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39259): Connection reset by peer (error 104)
2026-08-10T11:36:04Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (2195 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786361401922362-8267-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 11:36:04 dist-test-slave-srgv krb5kdc[30028](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 11:36:04 dist-test-slave-srgv krb5kdc[30028](info): set up 2 sockets
Aug 10 11:36:04 dist-test-slave-srgv krb5kdc[30028](info): commencing operation
krb5kdc: starting...
W20260810 11:36:05.878832 8267 mini_kdc.cc:121] Time spent starting KDC: real 1.126s user 0.004s sys 0.042s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786361401922362-8267-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786361401922362-8267-0/krb5kdc/test-user.keytab.
Aug 10 11:36:06 dist-test-slave-srgv krb5kdc[30028](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361766, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T11:36:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T11:36:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786361401922362-8267-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786361401922362-8267-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 11:36:06.226554 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786361401922362-8267-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786361401922362-8267-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786361401922362-8267-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39375
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786361401922362-8267-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33365
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39375
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786361401922362-8267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786361401922362-8267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786361401922362-8267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:36:06.593662 30042 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:36:06.594069 30042 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:36:06.594254 30042 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:36:06.601758 30042 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 11:36:06.601943 30042 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:36:06.602123 30042 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:36:06.602252 30042 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 11:36:06.602345 30042 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 11:36:06.612836 30042 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33365
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786361401922362-8267-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786361401922362-8267-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39375
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786361401922362-8267-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39375
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786361401922362-8267-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:36:06.614358 30042 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:36:06.615767 30042 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:36:06.629739 30047 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:36:06.629686 30050 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:36:06.629778 30048 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:36:06.630913 30042 server_base.cc:1034] running on GCE node
I20260810 11:36:06.631798 30042 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:36:06.633473 30042 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:36:06.634743 30042 hybrid_clock.cc:648] HybridClock initialized: now 1786361766634701 us; error 38 us; skew 500 ppm
I20260810 11:36:06.635185 30042 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:36:06 dist-test-slave-srgv krb5kdc[30028](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361766, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:36:06.639019 30042 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 11:36:06.640894 30042 webserver.cc:460] Webserver started at http://127.0.0.1:34709/ using document root <none> and password file <none>
I20260810 11:36:06.641768 30042 fs_manager.cc:359] Metadata directory not provided
I20260810 11:36:06.641916 30042 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:36:06.642261 30042 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:36:06.645694 30042 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786361401922362-8267-0/minicluster-data/master-0/data/instance:
uuid: "3344ca2bd8174e18b2121feef1344a90"
format_stamp: "Formatted at 2026-08-10 11:36:06 on dist-test-slave-srgv"
server_key: "e7f73b2ed1789bd203e55c0fe3d5d7b4"
server_key_iv: "3dce7ba9828fd96769ed4e4cac2442f8"
server_key_version: "encryptionkey@0"
I20260810 11:36:06.646560 30042 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786361401922362-8267-0/minicluster-data/master-0/wal/instance:
uuid: "3344ca2bd8174e18b2121feef1344a90"
format_stamp: "Formatted at 2026-08-10 11:36:06 on dist-test-slave-srgv"
server_key: "e7f73b2ed1789bd203e55c0fe3d5d7b4"
server_key_iv: "3dce7ba9828fd96769ed4e4cac2442f8"
server_key_version: "encryptionkey@0"
I20260810 11:36:06.652309 30042 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260810 11:36:06.656167 30057 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:36:06.657821 30042 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260810 11:36:06.658042 30042 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786361401922362-8267-0/minicluster-data/master-0/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786361401922362-8267-0/minicluster-data/master-0/wal
uuid: "3344ca2bd8174e18b2121feef1344a90"
format_stamp: "Formatted at 2026-08-10 11:36:06 on dist-test-slave-srgv"
server_key: "e7f73b2ed1789bd203e55c0fe3d5d7b4"
server_key_iv: "3dce7ba9828fd96769ed4e4cac2442f8"
server_key_version: "encryptionkey@0"
I20260810 11:36:06.658278 30042 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786361401922362-8267-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786361401922362-8267-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786361401922362-8267-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:36:06.672619 30042 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:36:06.677266 30042 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:36:06.677623 30042 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:36:06.699003 30042 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39375
I20260810 11:36:06.699041 30109 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39375 every 8 connection(s)
I20260810 11:36:06.700541 30042 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
I20260810 11:36:06.704612 30110 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:36:06.710971 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 30042
I20260810 11:36:06.711220 8267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786361401922362-8267-0/minicluster-data/master-0/wal/instance
I20260810 11:36:06.711685 8267 external_mini_cluster.cc:1428] Setting key cddd1104fb52b1f829cf7625c9fffd9e
I20260810 11:36:06.717953 30110 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3344ca2bd8174e18b2121feef1344a90: Bootstrap starting.
I20260810 11:36:06.722461 30110 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3344ca2bd8174e18b2121feef1344a90: Neither blocks nor log segments found. Creating new log.
I20260810 11:36:06.723917 30110 log.cc:824] T 00000000000000000000000000000000 P 3344ca2bd8174e18b2121feef1344a90: Log is configured to *not* fsync() on all Append() calls
I20260810 11:36:06.728214 30110 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3344ca2bd8174e18b2121feef1344a90: No bootstrap required, opened a new log
I20260810 11:36:06.729473 30113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:36:06.713832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.154:34673 (local address 127.0.0.1:39375)
0810 11:36:06.714820 (+ 988us) server_negotiation.cc:206] Beginning negotiation
0810 11:36:06.714844 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0810 11:36:06.714907 (+ 63us) server_negotiation.cc:407] Connection header received
0810 11:36:06.717537 (+ 2630us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:36:06.717576 (+ 39us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:36:06.718200 (+ 624us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:36:06.718723 (+ 523us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:36:06.719306 (+ 583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:36:06.721531 (+ 2225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:36:06.723233 (+ 1702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:36:06.724466 (+ 1233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:36:06.724756 (+ 290us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:36:06.726192 (+ 1436us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:36:06.726204 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:36:06.726235 (+ 31us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:36:06.727041 (+ 806us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:36:06.727148 (+ 107us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:36:06.727189 (+ 41us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:36:06.727423 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0810 11:36:06.727806 (+ 383us) server_negotiation.cc:299] Negotiation successful
0810 11:36:06.727993 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":704,"thread_start_us":455,"threads_started":1}
I20260810 11:36:06.734256 8267 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 11:36:06.737782 30110 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3344ca2bd8174e18b2121feef1344a90 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3344ca2bd8174e18b2121feef1344a90" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39375 } }
I20260810 11:36:06.738242 30110 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3344ca2bd8174e18b2121feef1344a90 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:36:06.738358 30110 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3344ca2bd8174e18b2121feef1344a90 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3344ca2bd8174e18b2121feef1344a90, State: Initialized, Role: FOLLOWER
I20260810 11:36:06.739223 30110 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3344ca2bd8174e18b2121feef1344a90 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3344ca2bd8174e18b2121feef1344a90" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39375 } }
I20260810 11:36:06.739615 30110 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3344ca2bd8174e18b2121feef1344a90 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 11:36:06.739810 30110 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3344ca2bd8174e18b2121feef1344a90 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 11:36:06.740026 30110 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3344ca2bd8174e18b2121feef1344a90 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:36:06.742854 30110 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3344ca2bd8174e18b2121feef1344a90 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3344ca2bd8174e18b2121feef1344a90" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39375 } }
I20260810 11:36:06.743458 30110 leader_election.cc:302] T 00000000000000000000000000000000 P 3344ca2bd8174e18b2121feef1344a90 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3344ca2bd8174e18b2121feef1344a90; no voters:
I20260810 11:36:06.744251 30110 leader_election.cc:288] T 00000000000000000000000000000000 P 3344ca2bd8174e18b2121feef1344a90 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 11:36:06.744416 30120 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3344ca2bd8174e18b2121feef1344a90 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:36:06.746680 30113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:36:06.739104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.154:52775 (local address 127.0.0.1:39375)
0810 11:36:06.739319 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0810 11:36:06.739327 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:36:06.739895 (+ 568us) server_negotiation.cc:407] Connection header received
0810 11:36:06.740145 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:36:06.740153 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:36:06.740239 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:36:06.740358 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:36:06.740937 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:36:06.741873 (+ 936us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:36:06.743425 (+ 1552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:36:06.744079 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:36:06.744254 (+ 175us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:36:06.745750 (+ 1496us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:36:06.745755 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:36:06.745759 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:36:06.745780 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:36:06.745823 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:36:06.745839 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:36:06.745967 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0810 11:36:06.746192 (+ 225us) server_negotiation.cc:299] Negotiation successful
0810 11:36:06.746254 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260810 11:36:06.747288 30110 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3344ca2bd8174e18b2121feef1344a90 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 11:36:06.749758 30120 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3344ca2bd8174e18b2121feef1344a90 [term 1 LEADER]: Becoming Leader. State: Replica: 3344ca2bd8174e18b2121feef1344a90, State: Running, Role: LEADER
I20260810 11:36:06.750681 30120 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3344ca2bd8174e18b2121feef1344a90 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3344ca2bd8174e18b2121feef1344a90" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39375 } }
I20260810 11:36:06.756057 30122 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3344ca2bd8174e18b2121feef1344a90 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3344ca2bd8174e18b2121feef1344a90. Latest consensus state: current_term: 1 leader_uuid: "3344ca2bd8174e18b2121feef1344a90" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3344ca2bd8174e18b2121feef1344a90" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39375 } } }
I20260810 11:36:06.756039 30121 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3344ca2bd8174e18b2121feef1344a90 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3344ca2bd8174e18b2121feef1344a90" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3344ca2bd8174e18b2121feef1344a90" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39375 } } }
I20260810 11:36:06.756611 30122 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3344ca2bd8174e18b2121feef1344a90 [sys.catalog]: This master's current role is: LEADER
I20260810 11:36:06.756634 30121 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3344ca2bd8174e18b2121feef1344a90 [sys.catalog]: This master's current role is: LEADER
I20260810 11:36:06.763115 30130 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 11:36:06.769058 30130 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 11:36:06.779798 30130 catalog_manager.cc:1347] Generated new cluster ID: 0a971420d653443db64b0f50973266da
I20260810 11:36:06.779973 30130 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 11:36:06.797494 30130 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 11:36:06.799053 30130 catalog_manager.cc:1504] Loading token signing keys...
I20260810 11:36:06.807384 30130 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3344ca2bd8174e18b2121feef1344a90: Generated new TSK 0
I20260810 11:36:06.808424 30130 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 11:36:06.826264 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 30042
W20260810 11:36:06.827730 30105 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.154:34673
W20260810 11:36:06.828665 8267 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 30042: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 11:36:06.828850 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 30042
W20260810 11:36:06.849061 30026 connection.cc:535] client connection to 127.0.0.1:39375 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39375): Connection reset by peer (error 104)
2026-08-10T11:36:06Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (2115 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786361401922362-8267-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 11:36:06 dist-test-slave-srgv krb5kdc[30143](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 11:36:06 dist-test-slave-srgv krb5kdc[30143](info): set up 2 sockets
Aug 10 11:36:06 dist-test-slave-srgv krb5kdc[30143](info): commencing operation
krb5kdc: starting...
W20260810 11:36:07.997718 8267 mini_kdc.cc:121] Time spent starting KDC: real 1.129s user 0.005s sys 0.039s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786361401922362-8267-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786361401922362-8267-0/krb5kdc/test-user.keytab.
Aug 10 11:36:08 dist-test-slave-srgv krb5kdc[30143](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361768, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T11:36:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T11:36:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786361401922362-8267-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786361401922362-8267-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 11:36:08.342693 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786361401922362-8267-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786361401922362-8267-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786361401922362-8267-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38823
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786361401922362-8267-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40183
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38823
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786361401922362-8267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786361401922362-8267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786361401922362-8267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:36:08.730084 30157 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:36:08.730422 30157 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:36:08.730646 30157 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:36:08.738346 30157 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 11:36:08.738519 30157 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:36:08.738613 30157 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:36:08.738713 30157 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 11:36:08.738781 30157 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 11:36:08.749686 30157 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40183
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786361401922362-8267-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786361401922362-8267-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38823
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786361401922362-8267-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38823
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786361401922362-8267-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:36:08.751181 30157 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:36:08.752676 30157 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:36:08.766417 30162 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:36:08.766387 30165 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:36:08.766345 30163 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:36:08.767452 30157 server_base.cc:1034] running on GCE node
I20260810 11:36:08.768313 30157 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:36:08.769909 30157 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:36:08.771138 30157 hybrid_clock.cc:648] HybridClock initialized: now 1786361768771097 us; error 34 us; skew 500 ppm
I20260810 11:36:08.771543 30157 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:36:08 dist-test-slave-srgv krb5kdc[30143](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361768, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:36:08.775139 30157 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 11:36:08.776974 30157 webserver.cc:460] Webserver started at http://127.0.0.1:39295/ using document root <none> and password file <none>
I20260810 11:36:08.777880 30157 fs_manager.cc:359] Metadata directory not provided
I20260810 11:36:08.778044 30157 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:36:08.778395 30157 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:36:08.781883 30157 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786361401922362-8267-0/minicluster-data/master-0/data/instance:
uuid: "309038293fef4c3a9bf8376e27844e3f"
format_stamp: "Formatted at 2026-08-10 11:36:08 on dist-test-slave-srgv"
server_key: "26feb2721149a6a8c65a3b508062ab4f"
server_key_iv: "020e94659e3f160603aa804b328a8af1"
server_key_version: "encryptionkey@0"
I20260810 11:36:08.782792 30157 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786361401922362-8267-0/minicluster-data/master-0/wal/instance:
uuid: "309038293fef4c3a9bf8376e27844e3f"
format_stamp: "Formatted at 2026-08-10 11:36:08 on dist-test-slave-srgv"
server_key: "26feb2721149a6a8c65a3b508062ab4f"
server_key_iv: "020e94659e3f160603aa804b328a8af1"
server_key_version: "encryptionkey@0"
I20260810 11:36:08.789371 30157 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260810 11:36:08.793470 30172 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:36:08.794978 30157 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.003s
I20260810 11:36:08.795224 30157 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786361401922362-8267-0/minicluster-data/master-0/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786361401922362-8267-0/minicluster-data/master-0/wal
uuid: "309038293fef4c3a9bf8376e27844e3f"
format_stamp: "Formatted at 2026-08-10 11:36:08 on dist-test-slave-srgv"
server_key: "26feb2721149a6a8c65a3b508062ab4f"
server_key_iv: "020e94659e3f160603aa804b328a8af1"
server_key_version: "encryptionkey@0"
I20260810 11:36:08.795470 30157 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786361401922362-8267-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786361401922362-8267-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786361401922362-8267-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:36:08.802215 30157 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:36:08.807147 30157 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:36:08.807534 30157 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:36:08.829586 30157 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38823
I20260810 11:36:08.829802 30224 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38823 every 8 connection(s)
I20260810 11:36:08.831274 30157 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
I20260810 11:36:08.835767 30225 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:36:08.837862 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 30157
I20260810 11:36:08.838073 8267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786361401922362-8267-0/minicluster-data/master-0/wal/instance
I20260810 11:36:08.838642 8267 external_mini_cluster.cc:1428] Setting key 0cd498583b638c82ec70117aaa488165
I20260810 11:36:08.847898 30228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:36:08.840834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.154:60001 (local address 127.0.0.1:38823)
0810 11:36:08.841924 (+ 1090us) server_negotiation.cc:206] Beginning negotiation
0810 11:36:08.841939 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 11:36:08.841996 (+ 57us) server_negotiation.cc:407] Connection header received
0810 11:36:08.843776 (+ 1780us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:36:08.843810 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:36:08.844411 (+ 601us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:36:08.844913 (+ 502us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:36:08.846124 (+ 1211us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:36:08.846131 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:36:08.846145 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:36:08.846162 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:36:08.846228 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:36:08.846249 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:36:08.846520 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0810 11:36:08.846666 (+ 146us) server_negotiation.cc:299] Negotiation successful
0810 11:36:08.846889 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":815,"thread_start_us":551,"threads_started":1}
I20260810 11:36:08.851872 30225 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 309038293fef4c3a9bf8376e27844e3f: Bootstrap starting.
I20260810 11:36:08.852178 8267 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 11:36:08.856215 30225 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 309038293fef4c3a9bf8376e27844e3f: Neither blocks nor log segments found. Creating new log.
I20260810 11:36:08.857834 30225 log.cc:824] T 00000000000000000000000000000000 P 309038293fef4c3a9bf8376e27844e3f: Log is configured to *not* fsync() on all Append() calls
I20260810 11:36:08.860863 30228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:36:08.858080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.154:40491 (local address 127.0.0.1:38823)
0810 11:36:08.858288 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0810 11:36:08.858296 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:36:08.858779 (+ 483us) server_negotiation.cc:407] Connection header received
0810 11:36:08.858999 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:36:08.859003 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:36:08.859057 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:36:08.859166 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:36:08.860177 (+ 1011us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:36:08.860182 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:36:08.860185 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:36:08.860205 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:36:08.860251 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:36:08.860266 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:36:08.860395 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0810 11:36:08.860515 (+ 120us) server_negotiation.cc:299] Negotiation successful
0810 11:36:08.860586 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260810 11:36:08.861397 30225 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 309038293fef4c3a9bf8376e27844e3f: No bootstrap required, opened a new log
I20260810 11:36:08.870304 30225 raft_consensus.cc:348] T 00000000000000000000000000000000 P 309038293fef4c3a9bf8376e27844e3f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "309038293fef4c3a9bf8376e27844e3f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38823 } }
I20260810 11:36:08.870923 30225 raft_consensus.cc:374] T 00000000000000000000000000000000 P 309038293fef4c3a9bf8376e27844e3f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:36:08.871088 30225 raft_consensus.cc:729] T 00000000000000000000000000000000 P 309038293fef4c3a9bf8376e27844e3f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 309038293fef4c3a9bf8376e27844e3f, State: Initialized, Role: FOLLOWER
I20260810 11:36:08.871841 30225 consensus_queue.cc:260] T 00000000000000000000000000000000 P 309038293fef4c3a9bf8376e27844e3f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "309038293fef4c3a9bf8376e27844e3f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38823 } }
I20260810 11:36:08.872129 30225 raft_consensus.cc:388] T 00000000000000000000000000000000 P 309038293fef4c3a9bf8376e27844e3f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 11:36:08.872309 30225 raft_consensus.cc:482] T 00000000000000000000000000000000 P 309038293fef4c3a9bf8376e27844e3f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 11:36:08.872498 30225 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 309038293fef4c3a9bf8376e27844e3f [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:36:08.875103 30225 raft_consensus.cc:504] T 00000000000000000000000000000000 P 309038293fef4c3a9bf8376e27844e3f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "309038293fef4c3a9bf8376e27844e3f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38823 } }
I20260810 11:36:08.875705 30225 leader_election.cc:302] T 00000000000000000000000000000000 P 309038293fef4c3a9bf8376e27844e3f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 309038293fef4c3a9bf8376e27844e3f; no voters:
I20260810 11:36:08.876338 30225 leader_election.cc:288] T 00000000000000000000000000000000 P 309038293fef4c3a9bf8376e27844e3f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 11:36:08.876538 30235 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 309038293fef4c3a9bf8376e27844e3f [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:36:08.877269 30235 raft_consensus.cc:686] T 00000000000000000000000000000000 P 309038293fef4c3a9bf8376e27844e3f [term 1 LEADER]: Becoming Leader. State: Replica: 309038293fef4c3a9bf8376e27844e3f, State: Running, Role: LEADER
I20260810 11:36:08.877996 30235 consensus_queue.cc:237] T 00000000000000000000000000000000 P 309038293fef4c3a9bf8376e27844e3f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "309038293fef4c3a9bf8376e27844e3f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38823 } }
I20260810 11:36:08.878754 30225 sys_catalog.cc:564] T 00000000000000000000000000000000 P 309038293fef4c3a9bf8376e27844e3f [sys.catalog]: configured and running, proceeding with master startup.
I20260810 11:36:08.881577 30236 sys_catalog.cc:455] T 00000000000000000000000000000000 P 309038293fef4c3a9bf8376e27844e3f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "309038293fef4c3a9bf8376e27844e3f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "309038293fef4c3a9bf8376e27844e3f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38823 } } }
I20260810 11:36:08.882123 30236 sys_catalog.cc:458] T 00000000000000000000000000000000 P 309038293fef4c3a9bf8376e27844e3f [sys.catalog]: This master's current role is: LEADER
I20260810 11:36:08.882308 30238 sys_catalog.cc:455] T 00000000000000000000000000000000 P 309038293fef4c3a9bf8376e27844e3f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 309038293fef4c3a9bf8376e27844e3f. Latest consensus state: current_term: 1 leader_uuid: "309038293fef4c3a9bf8376e27844e3f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "309038293fef4c3a9bf8376e27844e3f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38823 } } }
I20260810 11:36:08.882645 30238 sys_catalog.cc:458] T 00000000000000000000000000000000 P 309038293fef4c3a9bf8376e27844e3f [sys.catalog]: This master's current role is: LEADER
I20260810 11:36:08.883764 30246 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 11:36:08.889675 30246 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 11:36:08.900383 30246 catalog_manager.cc:1347] Generated new cluster ID: 389bdf6a95cd440e86b06f1f30fa58f0
I20260810 11:36:08.900563 30246 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 11:36:08.920959 30246 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 11:36:08.921574 30246 catalog_manager.cc:1504] Loading token signing keys...
I20260810 11:36:08.929204 30246 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 309038293fef4c3a9bf8376e27844e3f: Generated new TSK 0
I20260810 11:36:08.930438 30246 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 11:36:08.953284 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 30157
W20260810 11:36:08.954980 30220 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.154:60001
W20260810 11:36:08.955902 8267 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 30157: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 11:36:08.956084 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 30157
2026-08-10T11:36:08Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (2134 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786361401922362-8267-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 11:36:09 dist-test-slave-srgv krb5kdc[30259](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 11:36:09 dist-test-slave-srgv krb5kdc[30259](info): set up 2 sockets
Aug 10 11:36:09 dist-test-slave-srgv krb5kdc[30259](info): commencing operation
krb5kdc: starting...
W20260810 11:36:10.134384 8267 mini_kdc.cc:121] Time spent starting KDC: real 1.130s user 0.002s sys 0.042s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786361401922362-8267-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786361401922362-8267-0/krb5kdc/test-user.keytab.
Aug 10 11:36:10 dist-test-slave-srgv krb5kdc[30259](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361770, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T11:36:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T11:36:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786361401922362-8267-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786361401922362-8267-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 11:36:10.480465 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786361401922362-8267-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786361401922362-8267-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786361401922362-8267-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45537
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786361401922362-8267-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37357
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45537
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786361401922362-8267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786361401922362-8267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786361401922362-8267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:36:10.845350 30274 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:36:10.845813 30274 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:36:10.846002 30274 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:36:10.853789 30274 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 11:36:10.853940 30274 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:36:10.854060 30274 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:36:10.854167 30274 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 11:36:10.854257 30274 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 11:36:10.864773 30274 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37357
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786361401922362-8267-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786361401922362-8267-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45537
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786361401922362-8267-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45537
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786361401922362-8267-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:36:10.866276 30274 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:36:10.867656 30274 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:36:10.880589 30279 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:36:10.880519 30280 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:36:10.880627 30282 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:36:10.881752 30274 server_base.cc:1034] running on GCE node
I20260810 11:36:10.882705 30274 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:36:10.884483 30274 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:36:10.885721 30274 hybrid_clock.cc:648] HybridClock initialized: now 1786361770885685 us; error 45 us; skew 500 ppm
I20260810 11:36:10.886157 30274 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:36:10 dist-test-slave-srgv krb5kdc[30259](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361770, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:36:10.894600 30274 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 11:36:10.896476 30274 webserver.cc:460] Webserver started at http://127.0.0.1:37575/ using document root <none> and password file <none>
I20260810 11:36:10.897270 30274 fs_manager.cc:359] Metadata directory not provided
I20260810 11:36:10.897421 30274 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:36:10.897794 30274 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:36:10.901022 30274 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786361401922362-8267-0/minicluster-data/master-0/data/instance:
uuid: "245bf766dcf44f8ab12338d8903d4d67"
format_stamp: "Formatted at 2026-08-10 11:36:10 on dist-test-slave-srgv"
server_key: "f8a4b815d05c5e339a8fb15f1245442f"
server_key_iv: "dc1eee56b6ba142b6ca72220628a68ab"
server_key_version: "encryptionkey@0"
I20260810 11:36:10.901914 30274 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786361401922362-8267-0/minicluster-data/master-0/wal/instance:
uuid: "245bf766dcf44f8ab12338d8903d4d67"
format_stamp: "Formatted at 2026-08-10 11:36:10 on dist-test-slave-srgv"
server_key: "f8a4b815d05c5e339a8fb15f1245442f"
server_key_iv: "dc1eee56b6ba142b6ca72220628a68ab"
server_key_version: "encryptionkey@0"
I20260810 11:36:10.907804 30274 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260810 11:36:10.911772 30289 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:36:10.913244 30274 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 11:36:10.913509 30274 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786361401922362-8267-0/minicluster-data/master-0/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786361401922362-8267-0/minicluster-data/master-0/wal
uuid: "245bf766dcf44f8ab12338d8903d4d67"
format_stamp: "Formatted at 2026-08-10 11:36:10 on dist-test-slave-srgv"
server_key: "f8a4b815d05c5e339a8fb15f1245442f"
server_key_iv: "dc1eee56b6ba142b6ca72220628a68ab"
server_key_version: "encryptionkey@0"
I20260810 11:36:10.913715 30274 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786361401922362-8267-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786361401922362-8267-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786361401922362-8267-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:36:10.936791 30274 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:36:10.941540 30274 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:36:10.941890 30274 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:36:10.962463 30274 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45537
I20260810 11:36:10.962503 30342 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45537 every 8 connection(s)
I20260810 11:36:10.963965 30274 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
I20260810 11:36:10.964918 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 30274
I20260810 11:36:10.965116 8267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786361401922362-8267-0/minicluster-data/master-0/wal/instance
I20260810 11:36:10.965710 8267 external_mini_cluster.cc:1428] Setting key d28e923ffa767419b0a59b75386f6e05
I20260810 11:36:10.969235 30343 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 10 11:36:10 dist-test-slave-srgv krb5kdc[30259](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361770, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260810 11:36:10.983897 30343 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 245bf766dcf44f8ab12338d8903d4d67: Bootstrap starting.
I20260810 11:36:10.987541 30343 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 245bf766dcf44f8ab12338d8903d4d67: Neither blocks nor log segments found. Creating new log.
I20260810 11:36:10.988744 30343 log.cc:824] T 00000000000000000000000000000000 P 245bf766dcf44f8ab12338d8903d4d67: Log is configured to *not* fsync() on all Append() calls
I20260810 11:36:10.991456 30346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:36:10.967702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.154:50477 (local address 127.0.0.1:45537)
0810 11:36:10.968579 (+ 877us) server_negotiation.cc:206] Beginning negotiation
0810 11:36:10.968592 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 11:36:10.968636 (+ 44us) server_negotiation.cc:407] Connection header received
0810 11:36:10.970480 (+ 1844us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:36:10.970505 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:36:10.970927 (+ 422us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:36:10.971299 (+ 372us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:36:10.973302 (+ 2003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:36:10.974805 (+ 1503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:36:10.976754 (+ 1949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:36:10.977594 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:36:10.977738 (+ 144us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:36:10.983188 (+ 5450us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:36:10.983198 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:36:10.983216 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:36:10.983291 (+ 75us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:36:10.986750 (+ 3459us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:36:10.987731 (+ 981us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:36:10.987740 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:36:10.987746 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:36:10.987931 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:36:10.988495 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:36:10.988499 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:36:10.988501 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:36:10.989135 (+ 634us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:36:10.989541 (+ 406us) server_negotiation.cc:1035] Waiting for connection context
0810 11:36:10.990113 (+ 572us) server_negotiation.cc:299] Negotiation successful
0810 11:36:10.990449 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":618,"thread_start_us":368,"threads_started":1}
I20260810 11:36:10.992107 30343 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 245bf766dcf44f8ab12338d8903d4d67: No bootstrap required, opened a new log
I20260810 11:36:10.995841 8267 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 11:36:11.000862 30343 raft_consensus.cc:348] T 00000000000000000000000000000000 P 245bf766dcf44f8ab12338d8903d4d67 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "245bf766dcf44f8ab12338d8903d4d67" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45537 } }
I20260810 11:36:11.001355 30343 raft_consensus.cc:374] T 00000000000000000000000000000000 P 245bf766dcf44f8ab12338d8903d4d67 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:36:11.001533 30343 raft_consensus.cc:729] T 00000000000000000000000000000000 P 245bf766dcf44f8ab12338d8903d4d67 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 245bf766dcf44f8ab12338d8903d4d67, State: Initialized, Role: FOLLOWER
I20260810 11:36:11.002302 30343 consensus_queue.cc:260] T 00000000000000000000000000000000 P 245bf766dcf44f8ab12338d8903d4d67 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "245bf766dcf44f8ab12338d8903d4d67" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45537 } }
I20260810 11:36:11.002619 30343 raft_consensus.cc:388] T 00000000000000000000000000000000 P 245bf766dcf44f8ab12338d8903d4d67 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 11:36:11.002774 30343 raft_consensus.cc:482] T 00000000000000000000000000000000 P 245bf766dcf44f8ab12338d8903d4d67 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 11:36:11.002991 30343 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 245bf766dcf44f8ab12338d8903d4d67 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:36:11.006266 30343 raft_consensus.cc:504] T 00000000000000000000000000000000 P 245bf766dcf44f8ab12338d8903d4d67 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "245bf766dcf44f8ab12338d8903d4d67" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45537 } }
I20260810 11:36:11.006922 30343 leader_election.cc:302] T 00000000000000000000000000000000 P 245bf766dcf44f8ab12338d8903d4d67 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 245bf766dcf44f8ab12338d8903d4d67; no voters:
I20260810 11:36:11.007659 30343 leader_election.cc:288] T 00000000000000000000000000000000 P 245bf766dcf44f8ab12338d8903d4d67 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 11:36:11.007795 30354 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 245bf766dcf44f8ab12338d8903d4d67 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:36:11.009217 30354 raft_consensus.cc:686] T 00000000000000000000000000000000 P 245bf766dcf44f8ab12338d8903d4d67 [term 1 LEADER]: Becoming Leader. State: Replica: 245bf766dcf44f8ab12338d8903d4d67, State: Running, Role: LEADER
I20260810 11:36:11.010002 30354 consensus_queue.cc:237] T 00000000000000000000000000000000 P 245bf766dcf44f8ab12338d8903d4d67 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "245bf766dcf44f8ab12338d8903d4d67" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45537 } }
I20260810 11:36:11.010428 30343 sys_catalog.cc:564] T 00000000000000000000000000000000 P 245bf766dcf44f8ab12338d8903d4d67 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 11:36:11.013833 30355 sys_catalog.cc:455] T 00000000000000000000000000000000 P 245bf766dcf44f8ab12338d8903d4d67 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "245bf766dcf44f8ab12338d8903d4d67" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "245bf766dcf44f8ab12338d8903d4d67" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45537 } } }
I20260810 11:36:11.014370 30355 sys_catalog.cc:458] T 00000000000000000000000000000000 P 245bf766dcf44f8ab12338d8903d4d67 [sys.catalog]: This master's current role is: LEADER
I20260810 11:36:11.018958 30356 sys_catalog.cc:455] T 00000000000000000000000000000000 P 245bf766dcf44f8ab12338d8903d4d67 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 245bf766dcf44f8ab12338d8903d4d67. Latest consensus state: current_term: 1 leader_uuid: "245bf766dcf44f8ab12338d8903d4d67" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "245bf766dcf44f8ab12338d8903d4d67" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45537 } } }
I20260810 11:36:11.019433 30356 sys_catalog.cc:458] T 00000000000000000000000000000000 P 245bf766dcf44f8ab12338d8903d4d67 [sys.catalog]: This master's current role is: LEADER
I20260810 11:36:11.020174 30346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:36:11.000422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.154:36543 (local address 127.0.0.1:45537)
0810 11:36:11.000619 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0810 11:36:11.000626 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:36:11.001180 (+ 554us) server_negotiation.cc:407] Connection header received
0810 11:36:11.001353 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:36:11.001363 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:36:11.001478 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:36:11.001598 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:36:11.003840 (+ 2242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:36:11.004771 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:36:11.006583 (+ 1812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:36:11.007025 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:36:11.007168 (+ 143us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:36:11.010367 (+ 3199us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:36:11.010372 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:36:11.010376 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:36:11.010416 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:36:11.014572 (+ 4156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:36:11.016055 (+ 1483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:36:11.016065 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:36:11.016071 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:36:11.016342 (+ 271us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:36:11.017161 (+ 819us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:36:11.017170 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:36:11.017176 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:36:11.017748 (+ 572us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:36:11.019592 (+ 1844us) server_negotiation.cc:1035] Waiting for connection context
0810 11:36:11.019772 (+ 180us) server_negotiation.cc:299] Negotiation successful
0810 11:36:11.019926 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260810 11:36:11.024809 30360 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 11:36:11.030402 30360 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 11:36:11.040664 30360 catalog_manager.cc:1347] Generated new cluster ID: e5245f4fe42341b6aa6c0f0cfc87f52f
I20260810 11:36:11.040829 30360 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 11:36:11.059031 30360 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 11:36:11.060860 30360 catalog_manager.cc:1504] Loading token signing keys...
I20260810 11:36:11.068284 30360 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 245bf766dcf44f8ab12338d8903d4d67: Generated new TSK 0
I20260810 11:36:11.069556 30360 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 11:36:11.075524 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 30274
I20260810 11:36:11.077596 30338 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:36:11.173851 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 30274
W20260810 11:36:11.193833 30257 connection.cc:535] client connection to 127.0.0.1:45537 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:45537): Connection reset by peer (error 104)
2026-08-10T11:36:11Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (2209 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786361401922362-8267-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 11:36:11 dist-test-slave-srgv krb5kdc[30376](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 11:36:11 dist-test-slave-srgv krb5kdc[30376](info): set up 2 sockets
Aug 10 11:36:11 dist-test-slave-srgv krb5kdc[30376](info): commencing operation
krb5kdc: starting...
W20260810 11:36:12.353039 8267 mini_kdc.cc:121] Time spent starting KDC: real 1.139s user 0.005s sys 0.039s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786361401922362-8267-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786361401922362-8267-0/krb5kdc/test-user.keytab.
Aug 10 11:36:12 dist-test-slave-srgv krb5kdc[30376](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361772, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T11:36:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T11:36:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786361401922362-8267-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786361401922362-8267-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 11:36:12.698017 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786361401922362-8267-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786361401922362-8267-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786361401922362-8267-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46237
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786361401922362-8267-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44585
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46237
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786361401922362-8267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786361401922362-8267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786361401922362-8267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:36:13.068092 30389 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:36:13.068517 30389 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:36:13.068708 30389 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:36:13.076455 30389 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 11:36:13.076660 30389 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:36:13.076792 30389 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:36:13.076911 30389 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 11:36:13.076994 30389 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 11:36:13.087836 30389 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44585
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786361401922362-8267-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786361401922362-8267-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46237
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786361401922362-8267-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46237
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786361401922362-8267-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:36:13.089375 30389 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:36:13.090808 30389 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:36:13.103806 30395 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:36:13.103879 30394 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:36:13.103902 30398 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:36:13.104547 30389 server_base.cc:1034] running on GCE node
I20260810 11:36:13.105512 30389 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:36:13.107228 30389 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:36:13.108476 30389 hybrid_clock.cc:648] HybridClock initialized: now 1786361773108433 us; error 68 us; skew 500 ppm
I20260810 11:36:13.108888 30389 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:36:13 dist-test-slave-srgv krb5kdc[30376](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361773, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:36:13.112926 30389 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 11:36:13.114797 30389 webserver.cc:460] Webserver started at http://127.0.0.1:43087/ using document root <none> and password file <none>
I20260810 11:36:13.115556 30389 fs_manager.cc:359] Metadata directory not provided
I20260810 11:36:13.115729 30389 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:36:13.116070 30389 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:36:13.119549 30389 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786361401922362-8267-0/minicluster-data/master-0/data/instance:
uuid: "0c0574194e4940fb928b5280d068a9ce"
format_stamp: "Formatted at 2026-08-10 11:36:13 on dist-test-slave-srgv"
server_key: "8530ac89eb9eef6b35191e7506eb66e3"
server_key_iv: "4fe553d5336d32908ed127160fa0bc7f"
server_key_version: "encryptionkey@0"
I20260810 11:36:13.120450 30389 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786361401922362-8267-0/minicluster-data/master-0/wal/instance:
uuid: "0c0574194e4940fb928b5280d068a9ce"
format_stamp: "Formatted at 2026-08-10 11:36:13 on dist-test-slave-srgv"
server_key: "8530ac89eb9eef6b35191e7506eb66e3"
server_key_iv: "4fe553d5336d32908ed127160fa0bc7f"
server_key_version: "encryptionkey@0"
I20260810 11:36:13.126837 30389 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260810 11:36:13.131517 30406 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:36:13.133121 30389 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.000s
I20260810 11:36:13.133617 30389 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786361401922362-8267-0/minicluster-data/master-0/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786361401922362-8267-0/minicluster-data/master-0/wal
uuid: "0c0574194e4940fb928b5280d068a9ce"
format_stamp: "Formatted at 2026-08-10 11:36:13 on dist-test-slave-srgv"
server_key: "8530ac89eb9eef6b35191e7506eb66e3"
server_key_iv: "4fe553d5336d32908ed127160fa0bc7f"
server_key_version: "encryptionkey@0"
I20260810 11:36:13.133857 30389 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786361401922362-8267-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786361401922362-8267-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786361401922362-8267-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:36:13.149971 30389 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:36:13.155036 30389 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:36:13.155396 30389 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:36:13.177852 30389 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46237
I20260810 11:36:13.177875 30458 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46237 every 8 connection(s)
I20260810 11:36:13.179539 30389 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
I20260810 11:36:13.183851 30459 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:36:13.188941 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 30389
I20260810 11:36:13.189191 8267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786361401922362-8267-0/minicluster-data/master-0/wal/instance
I20260810 11:36:13.189697 8267 external_mini_cluster.cc:1428] Setting key af1a86a3c1b4c5411f33345f2cc14cc9
I20260810 11:36:13.197217 30459 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c0574194e4940fb928b5280d068a9ce: Bootstrap starting.
I20260810 11:36:13.201215 30459 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0c0574194e4940fb928b5280d068a9ce: Neither blocks nor log segments found. Creating new log.
I20260810 11:36:13.202708 30459 log.cc:824] T 00000000000000000000000000000000 P 0c0574194e4940fb928b5280d068a9ce: Log is configured to *not* fsync() on all Append() calls
I20260810 11:36:13.206341 30462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:36:13.191840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.154:45309 (local address 127.0.0.1:46237)
0810 11:36:13.192778 (+ 938us) server_negotiation.cc:206] Beginning negotiation
0810 11:36:13.192795 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0810 11:36:13.192850 (+ 55us) server_negotiation.cc:407] Connection header received
0810 11:36:13.195505 (+ 2655us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:36:13.195538 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:36:13.196123 (+ 585us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:36:13.196608 (+ 485us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:36:13.197144 (+ 536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:36:13.199209 (+ 2065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:36:13.200636 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:36:13.201833 (+ 1197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:36:13.202020 (+ 187us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:36:13.203347 (+ 1327us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:36:13.203365 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:36:13.203393 (+ 28us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:36:13.204090 (+ 697us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:36:13.204174 (+ 84us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:36:13.204249 (+ 75us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:36:13.204421 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0810 11:36:13.204751 (+ 330us) server_negotiation.cc:299] Negotiation successful
0810 11:36:13.204946 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":659,"thread_start_us":416,"threads_started":1}
I20260810 11:36:13.206534 30459 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c0574194e4940fb928b5280d068a9ce: No bootstrap required, opened a new log
I20260810 11:36:13.211155 8267 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 11:36:13.215485 30459 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0c0574194e4940fb928b5280d068a9ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c0574194e4940fb928b5280d068a9ce" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46237 } }
I20260810 11:36:13.216001 30459 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0c0574194e4940fb928b5280d068a9ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:36:13.216174 30459 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0c0574194e4940fb928b5280d068a9ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c0574194e4940fb928b5280d068a9ce, State: Initialized, Role: FOLLOWER
I20260810 11:36:13.217106 30459 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0c0574194e4940fb928b5280d068a9ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c0574194e4940fb928b5280d068a9ce" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46237 } }
I20260810 11:36:13.217473 30459 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0c0574194e4940fb928b5280d068a9ce [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 11:36:13.217653 30459 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0c0574194e4940fb928b5280d068a9ce [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 11:36:13.217855 30459 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0c0574194e4940fb928b5280d068a9ce [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:36:13.220877 30459 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0c0574194e4940fb928b5280d068a9ce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c0574194e4940fb928b5280d068a9ce" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46237 } }
I20260810 11:36:13.221616 30459 leader_election.cc:302] T 00000000000000000000000000000000 P 0c0574194e4940fb928b5280d068a9ce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0c0574194e4940fb928b5280d068a9ce; no voters:
I20260810 11:36:13.222525 30459 leader_election.cc:288] T 00000000000000000000000000000000 P 0c0574194e4940fb928b5280d068a9ce [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 11:36:13.222755 30469 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0c0574194e4940fb928b5280d068a9ce [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:36:13.224432 30462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:36:13.216228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.154:52745 (local address 127.0.0.1:46237)
0810 11:36:13.216447 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0810 11:36:13.216454 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:36:13.216801 (+ 347us) server_negotiation.cc:407] Connection header received
0810 11:36:13.216979 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:36:13.216984 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:36:13.217072 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:36:13.217222 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:36:13.217953 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:36:13.219187 (+ 1234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:36:13.220714 (+ 1527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:36:13.221489 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:36:13.221646 (+ 157us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:36:13.223225 (+ 1579us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:36:13.223237 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:36:13.223244 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:36:13.223278 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:36:13.223377 (+ 99us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:36:13.223415 (+ 38us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:36:13.223553 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0810 11:36:13.223987 (+ 434us) server_negotiation.cc:299] Negotiation successful
0810 11:36:13.224064 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260810 11:36:13.224684 30469 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0c0574194e4940fb928b5280d068a9ce [term 1 LEADER]: Becoming Leader. State: Replica: 0c0574194e4940fb928b5280d068a9ce, State: Running, Role: LEADER
I20260810 11:36:13.225317 30459 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0c0574194e4940fb928b5280d068a9ce [sys.catalog]: configured and running, proceeding with master startup.
I20260810 11:36:13.225449 30469 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0c0574194e4940fb928b5280d068a9ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c0574194e4940fb928b5280d068a9ce" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46237 } }
I20260810 11:36:13.228763 30470 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c0574194e4940fb928b5280d068a9ce [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0c0574194e4940fb928b5280d068a9ce" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c0574194e4940fb928b5280d068a9ce" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46237 } } }
I20260810 11:36:13.229314 30470 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c0574194e4940fb928b5280d068a9ce [sys.catalog]: This master's current role is: LEADER
I20260810 11:36:13.229156 30471 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c0574194e4940fb928b5280d068a9ce [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0c0574194e4940fb928b5280d068a9ce. Latest consensus state: current_term: 1 leader_uuid: "0c0574194e4940fb928b5280d068a9ce" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c0574194e4940fb928b5280d068a9ce" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46237 } } }
I20260810 11:36:13.229539 30471 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c0574194e4940fb928b5280d068a9ce [sys.catalog]: This master's current role is: LEADER
I20260810 11:36:13.231559 30477 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 11:36:13.237308 30477 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 11:36:13.247972 30477 catalog_manager.cc:1347] Generated new cluster ID: c0dcf3eb635341ee83bf2c2677f15f94
I20260810 11:36:13.248116 30477 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 11:36:13.265578 30477 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 11:36:13.267799 30477 catalog_manager.cc:1504] Loading token signing keys...
I20260810 11:36:13.280936 30477 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0c0574194e4940fb928b5280d068a9ce: Generated new TSK 0
I20260810 11:36:13.282044 30477 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 11:36:13.294905 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 30389
W20260810 11:36:13.296299 30454 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.154:45309
W20260810 11:36:13.297367 8267 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 30389: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 11:36:13.297583 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 30389
W20260810 11:36:13.317631 30374 connection.cc:535] client connection to 127.0.0.1:46237 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:46237): Connection reset by peer (error 104)
2026-08-10T11:36:13Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (2123 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786361401922362-8267-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 11:36:13 dist-test-slave-srgv krb5kdc[30491](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 11:36:13 dist-test-slave-srgv krb5kdc[30491](info): set up 2 sockets
Aug 10 11:36:13 dist-test-slave-srgv krb5kdc[30491](info): commencing operation
krb5kdc: starting...
W20260810 11:36:14.456734 8267 mini_kdc.cc:121] Time spent starting KDC: real 1.120s user 0.003s sys 0.039s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786361401922362-8267-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786361401922362-8267-0/krb5kdc/test-user.keytab.
Aug 10 11:36:14 dist-test-slave-srgv krb5kdc[30491](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361774, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T11:36:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T11:36:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786361401922362-8267-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786361401922362-8267-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 11:36:14.790562 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786361401922362-8267-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786361401922362-8267-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786361401922362-8267-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38375
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786361401922362-8267-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39853
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38375
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786361401922362-8267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786361401922362-8267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786361401922362-8267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:36:15.163594 30504 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:36:15.163923 30504 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:36:15.164126 30504 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:36:15.171897 30504 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 11:36:15.172050 30504 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:36:15.172158 30504 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:36:15.172286 30504 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 11:36:15.172370 30504 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 11:36:15.183143 30504 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39853
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786361401922362-8267-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786361401922362-8267-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38375
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786361401922362-8267-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38375
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786361401922362-8267-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:36:15.184782 30504 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:36:15.186339 30504 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:36:15.199357 30509 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:36:15.199429 30510 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:36:15.199473 30512 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:36:15.200687 30504 server_base.cc:1034] running on GCE node
I20260810 11:36:15.201709 30504 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:36:15.203377 30504 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:36:15.204635 30504 hybrid_clock.cc:648] HybridClock initialized: now 1786361775204588 us; error 65 us; skew 500 ppm
I20260810 11:36:15.205037 30504 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:36:15 dist-test-slave-srgv krb5kdc[30491](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361775, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:36:15.208757 30504 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 11:36:15.210641 30504 webserver.cc:460] Webserver started at http://127.0.0.1:41271/ using document root <none> and password file <none>
I20260810 11:36:15.211421 30504 fs_manager.cc:359] Metadata directory not provided
I20260810 11:36:15.211567 30504 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:36:15.211930 30504 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:36:15.215447 30504 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786361401922362-8267-0/minicluster-data/master-0/data/instance:
uuid: "d9fe59c825c6460584870de558807f43"
format_stamp: "Formatted at 2026-08-10 11:36:15 on dist-test-slave-srgv"
server_key: "3bd1150e878d42101fe890b16a64a390"
server_key_iv: "ab86f557f79d1284ccc6359dc94fbff7"
server_key_version: "encryptionkey@0"
I20260810 11:36:15.216413 30504 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786361401922362-8267-0/minicluster-data/master-0/wal/instance:
uuid: "d9fe59c825c6460584870de558807f43"
format_stamp: "Formatted at 2026-08-10 11:36:15 on dist-test-slave-srgv"
server_key: "3bd1150e878d42101fe890b16a64a390"
server_key_iv: "ab86f557f79d1284ccc6359dc94fbff7"
server_key_version: "encryptionkey@0"
I20260810 11:36:15.222981 30504 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260810 11:36:15.227376 30519 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:36:15.228962 30504 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 11:36:15.229143 30504 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786361401922362-8267-0/minicluster-data/master-0/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786361401922362-8267-0/minicluster-data/master-0/wal
uuid: "d9fe59c825c6460584870de558807f43"
format_stamp: "Formatted at 2026-08-10 11:36:15 on dist-test-slave-srgv"
server_key: "3bd1150e878d42101fe890b16a64a390"
server_key_iv: "ab86f557f79d1284ccc6359dc94fbff7"
server_key_version: "encryptionkey@0"
I20260810 11:36:15.229509 30504 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786361401922362-8267-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786361401922362-8267-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786361401922362-8267-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:36:15.236433 30504 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:36:15.241675 30504 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:36:15.242024 30504 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:36:15.263952 30504 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38375
I20260810 11:36:15.264029 30574 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38375 every 8 connection(s)
I20260810 11:36:15.265748 30504 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
I20260810 11:36:15.270100 30575 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:36:15.275161 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 30504
I20260810 11:36:15.275333 8267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786361401922362-8267-0/minicluster-data/master-0/wal/instance
I20260810 11:36:15.275810 8267 external_mini_cluster.cc:1428] Setting key 11fb3f24ada7683a35c2ba9b404e89ba
I20260810 11:36:15.282864 30575 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d9fe59c825c6460584870de558807f43: Bootstrap starting.
I20260810 11:36:15.285007 30578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:36:15.277974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.154:52929 (local address 127.0.0.1:38375)
0810 11:36:15.278901 (+ 927us) server_negotiation.cc:206] Beginning negotiation
0810 11:36:15.278917 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0810 11:36:15.278964 (+ 47us) server_negotiation.cc:407] Connection header received
0810 11:36:15.281045 (+ 2081us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:36:15.281074 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:36:15.281560 (+ 486us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:36:15.281935 (+ 375us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:36:15.283149 (+ 1214us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:36:15.283156 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:36:15.283170 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:36:15.283187 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:36:15.283291 (+ 104us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:36:15.283316 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:36:15.283517 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0810 11:36:15.283653 (+ 136us) server_negotiation.cc:299] Negotiation successful
0810 11:36:15.283892 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":683,"thread_start_us":434,"threads_started":1}
I20260810 11:36:15.286825 30575 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d9fe59c825c6460584870de558807f43: Neither blocks nor log segments found. Creating new log.
I20260810 11:36:15.288169 30575 log.cc:824] T 00000000000000000000000000000000 P d9fe59c825c6460584870de558807f43: Log is configured to *not* fsync() on all Append() calls
I20260810 11:36:15.289621 8267 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 11:36:15.291628 30575 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d9fe59c825c6460584870de558807f43: No bootstrap required, opened a new log
I20260810 11:36:15.297580 30578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:36:15.294611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.154:43499 (local address 127.0.0.1:38375)
0810 11:36:15.294846 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0810 11:36:15.294857 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 11:36:15.295130 (+ 273us) server_negotiation.cc:407] Connection header received
0810 11:36:15.295375 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:36:15.295383 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:36:15.295461 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:36:15.295584 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:36:15.296691 (+ 1107us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:36:15.296700 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:36:15.296705 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:36:15.296735 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:36:15.296786 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:36:15.296803 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:36:15.296962 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0810 11:36:15.297098 (+ 136us) server_negotiation.cc:299] Negotiation successful
0810 11:36:15.297187 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260810 11:36:15.301605 30575 raft_consensus.cc:348] T 00000000000000000000000000000000 P d9fe59c825c6460584870de558807f43 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9fe59c825c6460584870de558807f43" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38375 } }
I20260810 11:36:15.302085 30575 raft_consensus.cc:374] T 00000000000000000000000000000000 P d9fe59c825c6460584870de558807f43 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:36:15.302213 30575 raft_consensus.cc:729] T 00000000000000000000000000000000 P d9fe59c825c6460584870de558807f43 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9fe59c825c6460584870de558807f43, State: Initialized, Role: FOLLOWER
I20260810 11:36:15.302845 30575 consensus_queue.cc:260] T 00000000000000000000000000000000 P d9fe59c825c6460584870de558807f43 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9fe59c825c6460584870de558807f43" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38375 } }
I20260810 11:36:15.303122 30575 raft_consensus.cc:388] T 00000000000000000000000000000000 P d9fe59c825c6460584870de558807f43 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 11:36:15.303294 30575 raft_consensus.cc:482] T 00000000000000000000000000000000 P d9fe59c825c6460584870de558807f43 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 11:36:15.303479 30575 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d9fe59c825c6460584870de558807f43 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:36:15.306607 30575 raft_consensus.cc:504] T 00000000000000000000000000000000 P d9fe59c825c6460584870de558807f43 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9fe59c825c6460584870de558807f43" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38375 } }
I20260810 11:36:15.307269 30575 leader_election.cc:302] T 00000000000000000000000000000000 P d9fe59c825c6460584870de558807f43 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d9fe59c825c6460584870de558807f43; no voters:
I20260810 11:36:15.307934 30575 leader_election.cc:288] T 00000000000000000000000000000000 P d9fe59c825c6460584870de558807f43 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 11:36:15.308126 30585 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d9fe59c825c6460584870de558807f43 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:36:15.308930 30585 raft_consensus.cc:686] T 00000000000000000000000000000000 P d9fe59c825c6460584870de558807f43 [term 1 LEADER]: Becoming Leader. State: Replica: d9fe59c825c6460584870de558807f43, State: Running, Role: LEADER
I20260810 11:36:15.309732 30585 consensus_queue.cc:237] T 00000000000000000000000000000000 P d9fe59c825c6460584870de558807f43 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9fe59c825c6460584870de558807f43" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38375 } }
I20260810 11:36:15.310359 30575 sys_catalog.cc:564] T 00000000000000000000000000000000 P d9fe59c825c6460584870de558807f43 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 11:36:15.313091 30586 sys_catalog.cc:455] T 00000000000000000000000000000000 P d9fe59c825c6460584870de558807f43 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d9fe59c825c6460584870de558807f43" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9fe59c825c6460584870de558807f43" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38375 } } }
I20260810 11:36:15.313727 30586 sys_catalog.cc:458] T 00000000000000000000000000000000 P d9fe59c825c6460584870de558807f43 [sys.catalog]: This master's current role is: LEADER
I20260810 11:36:15.313035 30587 sys_catalog.cc:455] T 00000000000000000000000000000000 P d9fe59c825c6460584870de558807f43 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d9fe59c825c6460584870de558807f43. Latest consensus state: current_term: 1 leader_uuid: "d9fe59c825c6460584870de558807f43" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9fe59c825c6460584870de558807f43" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38375 } } }
I20260810 11:36:15.317708 30587 sys_catalog.cc:458] T 00000000000000000000000000000000 P d9fe59c825c6460584870de558807f43 [sys.catalog]: This master's current role is: LEADER
I20260810 11:36:15.318657 30593 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 11:36:15.323577 30593 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 11:36:15.333241 30593 catalog_manager.cc:1347] Generated new cluster ID: d07c3a67a6c547858bc024c217a6b3f4
I20260810 11:36:15.333431 30593 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 11:36:15.368254 30593 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 11:36:15.368767 30593 catalog_manager.cc:1504] Loading token signing keys...
I20260810 11:36:15.376600 30593 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d9fe59c825c6460584870de558807f43: Generated new TSK 0
I20260810 11:36:15.377759 30593 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 11:36:15.393968 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 30504
W20260810 11:36:15.395406 30569 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.154:52929
W20260810 11:36:15.396255 8267 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 30504: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 11:36:15.396449 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 30504
2026-08-10T11:36:15Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (2098 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786361401922362-8267-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 11:36:15 dist-test-slave-srgv krb5kdc[30606](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 11:36:15 dist-test-slave-srgv krb5kdc[30606](info): set up 2 sockets
Aug 10 11:36:15 dist-test-slave-srgv krb5kdc[30606](info): commencing operation
krb5kdc: starting...
W20260810 11:36:16.578171 8267 mini_kdc.cc:121] Time spent starting KDC: real 1.142s user 0.000s sys 0.044s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786361401922362-8267-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786361401922362-8267-0/krb5kdc/test-user.keytab.
Aug 10 11:36:16 dist-test-slave-srgv krb5kdc[30606](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361776, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T11:36:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T11:36:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.18.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.18.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.18.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.254.keytab.
Entry for principal kudu/127.8.18.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.254.keytab.
I20260810 11:36:16.910169 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786361401922362-8267-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786361401922362-8267-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786361401922362-8267-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.18.254:43503
--webserver_interface=127.8.18.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.254.keytab
--principal=kudu/127.8.18.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.18.212:45167
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.18.254:43503
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786361401922362-8267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786361401922362-8267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786361401922362-8267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:36:17.282806 30620 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:36:17.283149 30620 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:36:17.283257 30620 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:36:17.291002 30620 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 11:36:17.291148 30620 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:36:17.291208 30620 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:36:17.291303 30620 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 11:36:17.291353 30620 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 11:36:17.301910 30620 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.18.212:45167
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786361401922362-8267-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786361401922362-8267-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.18.254:43503
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.254.keytab
--principal=kudu/127.8.18.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.18.254:43503
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.18.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786361401922362-8267-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:36:17.303426 30620 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:36:17.305078 30620 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:36:17.318085 30625 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:36:17.318152 30627 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:36:17.318264 30630 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:36:17.319451 30620 server_base.cc:1034] running on GCE node
I20260810 11:36:17.320433 30620 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:36:17.322114 30620 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:36:17.323372 30620 hybrid_clock.cc:648] HybridClock initialized: now 1786361777323322 us; error 38 us; skew 500 ppm
I20260810 11:36:17.323792 30620 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:36:17 dist-test-slave-srgv krb5kdc[30606](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361777, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:36:17.332046 30620 init.cc:377] Logged in from keytab as kudu/127.8.18.254@KRBTEST.COM (short username kudu)
I20260810 11:36:17.333889 30620 webserver.cc:460] Webserver started at http://127.8.18.254:43313/ using document root <none> and password file <none>
I20260810 11:36:17.334621 30620 fs_manager.cc:359] Metadata directory not provided
I20260810 11:36:17.334734 30620 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:36:17.335074 30620 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:36:17.338515 30620 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786361401922362-8267-0/minicluster-data/master-0/data/instance:
uuid: "d402f22956dc4721a2b5d5295f43ee22"
format_stamp: "Formatted at 2026-08-10 11:36:17 on dist-test-slave-srgv"
server_key: "59f1de18df42f13be8a5da286123ea2c"
server_key_iv: "f108249dd91edce1508e9a68f4c64736"
server_key_version: "encryptionkey@0"
I20260810 11:36:17.339327 30620 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786361401922362-8267-0/minicluster-data/master-0/wal/instance:
uuid: "d402f22956dc4721a2b5d5295f43ee22"
format_stamp: "Formatted at 2026-08-10 11:36:17 on dist-test-slave-srgv"
server_key: "59f1de18df42f13be8a5da286123ea2c"
server_key_iv: "f108249dd91edce1508e9a68f4c64736"
server_key_version: "encryptionkey@0"
I20260810 11:36:17.345245 30620 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.005s
I20260810 11:36:17.349256 30637 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:36:17.350924 30620 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 11:36:17.351179 30620 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786361401922362-8267-0/minicluster-data/master-0/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786361401922362-8267-0/minicluster-data/master-0/wal
uuid: "d402f22956dc4721a2b5d5295f43ee22"
format_stamp: "Formatted at 2026-08-10 11:36:17 on dist-test-slave-srgv"
server_key: "59f1de18df42f13be8a5da286123ea2c"
server_key_iv: "f108249dd91edce1508e9a68f4c64736"
server_key_version: "encryptionkey@0"
I20260810 11:36:17.351436 30620 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786361401922362-8267-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786361401922362-8267-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786361401922362-8267-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:36:17.373704 30620 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:36:17.378856 30620 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:36:17.379215 30620 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:36:17.400487 30620 rpc_server.cc:307] RPC server started. Bound to: 127.8.18.254:43503
I20260810 11:36:17.400535 30691 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.18.254:43503 every 8 connection(s)
I20260810 11:36:17.402102 30620 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
I20260810 11:36:17.405293 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 30620
I20260810 11:36:17.405552 8267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786361401922362-8267-0/minicluster-data/master-0/wal/instance
I20260810 11:36:17.406032 8267 external_mini_cluster.cc:1428] Setting key 73dbf432f568db11c28ff0024b09c006
I20260810 11:36:17.406366 30692 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:36:17.422195 30692 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d402f22956dc4721a2b5d5295f43ee22: Bootstrap starting.
Aug 10 11:36:17 dist-test-slave-srgv krb5kdc[30606](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361776, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.18.254@KRBTEST.COM
I20260810 11:36:17.426534 30692 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d402f22956dc4721a2b5d5295f43ee22: Neither blocks nor log segments found. Creating new log.
I20260810 11:36:17.427999 30692 log.cc:824] T 00000000000000000000000000000000 P d402f22956dc4721a2b5d5295f43ee22: Log is configured to *not* fsync() on all Append() calls
I20260810 11:36:17.431893 30692 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d402f22956dc4721a2b5d5295f43ee22: No bootstrap required, opened a new log
I20260810 11:36:17.434046 30695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:36:17.408180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38567 (local address 127.8.18.254:43503)
0810 11:36:17.409287 (+ 1107us) server_negotiation.cc:206] Beginning negotiation
0810 11:36:17.409308 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0810 11:36:17.409737 (+ 429us) server_negotiation.cc:407] Connection header received
0810 11:36:17.411692 (+ 1955us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:36:17.411725 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:36:17.412330 (+ 605us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:36:17.412825 (+ 495us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:36:17.414287 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:36:17.416358 (+ 2071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:36:17.417829 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:36:17.419017 (+ 1188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:36:17.419271 (+ 254us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:36:17.423952 (+ 4681us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:36:17.423967 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:36:17.423990 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:36:17.424069 (+ 79us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:36:17.428334 (+ 4265us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:36:17.429585 (+ 1251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:36:17.429594 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:36:17.429602 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:36:17.429826 (+ 224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:36:17.430318 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:36:17.430325 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:36:17.430331 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:36:17.431093 (+ 762us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:36:17.431497 (+ 404us) server_negotiation.cc:1035] Waiting for connection context
0810 11:36:17.431950 (+ 453us) server_negotiation.cc:299] Negotiation successful
0810 11:36:17.432387 (+ 437us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":803,"thread_start_us":473,"threads_started":1}
I20260810 11:36:17.438597 8267 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 11:36:17.441926 30692 raft_consensus.cc:348] T 00000000000000000000000000000000 P d402f22956dc4721a2b5d5295f43ee22 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d402f22956dc4721a2b5d5295f43ee22" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 43503 } }
I20260810 11:36:17.442365 30692 raft_consensus.cc:374] T 00000000000000000000000000000000 P d402f22956dc4721a2b5d5295f43ee22 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:36:17.442472 30692 raft_consensus.cc:729] T 00000000000000000000000000000000 P d402f22956dc4721a2b5d5295f43ee22 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d402f22956dc4721a2b5d5295f43ee22, State: Initialized, Role: FOLLOWER
I20260810 11:36:17.443184 30692 consensus_queue.cc:260] T 00000000000000000000000000000000 P d402f22956dc4721a2b5d5295f43ee22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d402f22956dc4721a2b5d5295f43ee22" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 43503 } }
I20260810 11:36:17.443593 30692 raft_consensus.cc:388] T 00000000000000000000000000000000 P d402f22956dc4721a2b5d5295f43ee22 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 11:36:17.443771 30692 raft_consensus.cc:482] T 00000000000000000000000000000000 P d402f22956dc4721a2b5d5295f43ee22 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 11:36:17.444032 30692 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d402f22956dc4721a2b5d5295f43ee22 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:36:17.446666 30692 raft_consensus.cc:504] T 00000000000000000000000000000000 P d402f22956dc4721a2b5d5295f43ee22 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d402f22956dc4721a2b5d5295f43ee22" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 43503 } }
I20260810 11:36:17.447326 30692 leader_election.cc:302] T 00000000000000000000000000000000 P d402f22956dc4721a2b5d5295f43ee22 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d402f22956dc4721a2b5d5295f43ee22; no voters:
I20260810 11:36:17.448112 30692 leader_election.cc:288] T 00000000000000000000000000000000 P d402f22956dc4721a2b5d5295f43ee22 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 11:36:17.448239 30702 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d402f22956dc4721a2b5d5295f43ee22 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:36:17.449713 30702 raft_consensus.cc:686] T 00000000000000000000000000000000 P d402f22956dc4721a2b5d5295f43ee22 [term 1 LEADER]: Becoming Leader. State: Replica: d402f22956dc4721a2b5d5295f43ee22, State: Running, Role: LEADER
I20260810 11:36:17.450481 30702 consensus_queue.cc:237] T 00000000000000000000000000000000 P d402f22956dc4721a2b5d5295f43ee22 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d402f22956dc4721a2b5d5295f43ee22" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 43503 } }
I20260810 11:36:17.450938 30692 sys_catalog.cc:564] T 00000000000000000000000000000000 P d402f22956dc4721a2b5d5295f43ee22 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 11:36:17.454234 30703 sys_catalog.cc:455] T 00000000000000000000000000000000 P d402f22956dc4721a2b5d5295f43ee22 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d402f22956dc4721a2b5d5295f43ee22" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d402f22956dc4721a2b5d5295f43ee22" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 43503 } } }
I20260810 11:36:17.454785 30703 sys_catalog.cc:458] T 00000000000000000000000000000000 P d402f22956dc4721a2b5d5295f43ee22 [sys.catalog]: This master's current role is: LEADER
I20260810 11:36:17.456211 30709 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 11:36:17.456032 30704 sys_catalog.cc:455] T 00000000000000000000000000000000 P d402f22956dc4721a2b5d5295f43ee22 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d402f22956dc4721a2b5d5295f43ee22. Latest consensus state: current_term: 1 leader_uuid: "d402f22956dc4721a2b5d5295f43ee22" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d402f22956dc4721a2b5d5295f43ee22" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 43503 } } }
I20260810 11:36:17.456545 30704 sys_catalog.cc:458] T 00000000000000000000000000000000 P d402f22956dc4721a2b5d5295f43ee22 [sys.catalog]: This master's current role is: LEADER
I20260810 11:36:17.463709 30695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:36:17.443567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52539 (local address 127.8.18.254:43503)
0810 11:36:17.443782 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0810 11:36:17.443791 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 11:36:17.444345 (+ 554us) server_negotiation.cc:407] Connection header received
0810 11:36:17.444580 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:36:17.444585 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:36:17.444669 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:36:17.444804 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:36:17.446516 (+ 1712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:36:17.447424 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:36:17.449144 (+ 1720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:36:17.449941 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:36:17.450077 (+ 136us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:36:17.453226 (+ 3149us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:36:17.453235 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:36:17.453240 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:36:17.453295 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:36:17.457803 (+ 4508us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:36:17.459139 (+ 1336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:36:17.459147 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:36:17.459151 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:36:17.459312 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:36:17.460037 (+ 725us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:36:17.460045 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:36:17.460049 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:36:17.460646 (+ 597us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:36:17.462607 (+ 1961us) server_negotiation.cc:1035] Waiting for connection context
0810 11:36:17.462869 (+ 262us) server_negotiation.cc:299] Negotiation successful
0810 11:36:17.463147 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260810 11:36:17.470209 30709 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 11:36:17.481194 30709 catalog_manager.cc:1347] Generated new cluster ID: 2447eb4d1163480ea86ac5464d8747f5
I20260810 11:36:17.481351 30709 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 11:36:17.488814 30709 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 11:36:17.490737 30709 catalog_manager.cc:1504] Loading token signing keys...
I20260810 11:36:17.507404 30709 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d402f22956dc4721a2b5d5295f43ee22: Generated new TSK 0
I20260810 11:36:17.508388 30709 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 11:36:17.521978 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 30620
I20260810 11:36:17.523559 30687 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:36:17.620509 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 30620
W20260810 11:36:17.640731 30604 connection.cc:535] client connection to 127.8.18.254:43503 recv error: Network error: failed to read from TLS socket (remote: 127.8.18.254:43503): Connection reset by peer (error 104)
2026-08-10T11:36:17Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (2224 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786361401922362-8267-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 11:36:17 dist-test-slave-srgv krb5kdc[30725](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 11:36:17 dist-test-slave-srgv krb5kdc[30725](info): set up 2 sockets
Aug 10 11:36:17 dist-test-slave-srgv krb5kdc[30725](info): commencing operation
krb5kdc: starting...
W20260810 11:36:18.784518 8267 mini_kdc.cc:121] Time spent starting KDC: real 1.124s user 0.003s sys 0.040s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786361401922362-8267-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786361401922362-8267-0/krb5kdc/test-user.keytab.
Aug 10 11:36:18 dist-test-slave-srgv krb5kdc[30725](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361778, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T11:36:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T11:36:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.18.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.18.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.18.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.254.keytab.
Entry for principal kudu/127.8.18.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.254.keytab.
I20260810 11:36:19.117215 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786361401922362-8267-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786361401922362-8267-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786361401922362-8267-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.18.254:41795
--webserver_interface=127.8.18.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.254.keytab
--principal=kudu/127.8.18.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.18.212:37287
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.18.254:41795
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786361401922362-8267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786361401922362-8267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786361401922362-8267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:36:19.482538 30739 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:36:19.482949 30739 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:36:19.483131 30739 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:36:19.490914 30739 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 11:36:19.491113 30739 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:36:19.491241 30739 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:36:19.491357 30739 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 11:36:19.491439 30739 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 11:36:19.502013 30739 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.18.212:37287
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786361401922362-8267-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786361401922362-8267-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.18.254:41795
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.254.keytab
--principal=kudu/127.8.18.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.18.254:41795
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.18.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786361401922362-8267-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:36:19.503487 30739 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:36:19.504961 30739 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:36:19.517889 30744 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:36:19.517796 30745 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:36:19.517758 30747 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:36:19.518844 30739 server_base.cc:1034] running on GCE node
I20260810 11:36:19.519645 30739 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:36:19.521328 30739 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:36:19.522569 30739 hybrid_clock.cc:648] HybridClock initialized: now 1786361779522523 us; error 40 us; skew 500 ppm
I20260810 11:36:19.522922 30739 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:36:19 dist-test-slave-srgv krb5kdc[30725](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361779, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:36:19.526271 30739 init.cc:377] Logged in from keytab as kudu/127.8.18.254@KRBTEST.COM (short username kudu)
I20260810 11:36:19.527926 30739 webserver.cc:460] Webserver started at http://127.8.18.254:44263/ using document root <none> and password file <none>
I20260810 11:36:19.528683 30739 fs_manager.cc:359] Metadata directory not provided
I20260810 11:36:19.528831 30739 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:36:19.529177 30739 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:36:19.532447 30739 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786361401922362-8267-0/minicluster-data/master-0/data/instance:
uuid: "9102f493b1014764bf5dd0754073d439"
format_stamp: "Formatted at 2026-08-10 11:36:19 on dist-test-slave-srgv"
server_key: "b162d93be9cfeb3e95bba8c5efab5ca1"
server_key_iv: "b75ba9038fe22bf6bc99aee11c7047ca"
server_key_version: "encryptionkey@0"
I20260810 11:36:19.533267 30739 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786361401922362-8267-0/minicluster-data/master-0/wal/instance:
uuid: "9102f493b1014764bf5dd0754073d439"
format_stamp: "Formatted at 2026-08-10 11:36:19 on dist-test-slave-srgv"
server_key: "b162d93be9cfeb3e95bba8c5efab5ca1"
server_key_iv: "b75ba9038fe22bf6bc99aee11c7047ca"
server_key_version: "encryptionkey@0"
I20260810 11:36:19.538851 30739 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.002s
I20260810 11:36:19.542711 30754 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:36:19.544212 30739 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 11:36:19.544418 30739 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786361401922362-8267-0/minicluster-data/master-0/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786361401922362-8267-0/minicluster-data/master-0/wal
uuid: "9102f493b1014764bf5dd0754073d439"
format_stamp: "Formatted at 2026-08-10 11:36:19 on dist-test-slave-srgv"
server_key: "b162d93be9cfeb3e95bba8c5efab5ca1"
server_key_iv: "b75ba9038fe22bf6bc99aee11c7047ca"
server_key_version: "encryptionkey@0"
I20260810 11:36:19.544656 30739 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786361401922362-8267-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786361401922362-8267-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786361401922362-8267-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:36:19.562841 30739 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:36:19.567353 30739 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:36:19.567678 30739 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:36:19.589991 30739 rpc_server.cc:307] RPC server started. Bound to: 127.8.18.254:41795
I20260810 11:36:19.590040 30807 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.18.254:41795 every 8 connection(s)
I20260810 11:36:19.591529 30739 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
I20260810 11:36:19.591812 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 30739
I20260810 11:36:19.591998 8267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786361401922362-8267-0/minicluster-data/master-0/wal/instance
I20260810 11:36:19.592470 8267 external_mini_cluster.cc:1428] Setting key 9b48f311c3e5c114bf9182efc581768b
I20260810 11:36:19.596340 30808 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:36:19.607604 30811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:36:19.594584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51937 (local address 127.8.18.254:41795)
0810 11:36:19.595459 (+ 875us) server_negotiation.cc:206] Beginning negotiation
0810 11:36:19.595472 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 11:36:19.595515 (+ 43us) server_negotiation.cc:407] Connection header received
0810 11:36:19.597327 (+ 1812us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:36:19.597351 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:36:19.597827 (+ 476us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:36:19.598168 (+ 341us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:36:19.598899 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:36:19.600272 (+ 1373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:36:19.602260 (+ 1988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:36:19.603000 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:36:19.603140 (+ 140us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:36:19.604736 (+ 1596us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:36:19.604743 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:36:19.604758 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:36:19.605564 (+ 806us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:36:19.605682 (+ 118us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:36:19.605718 (+ 36us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:36:19.605855 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0810 11:36:19.606261 (+ 406us) server_negotiation.cc:299] Negotiation successful
0810 11:36:19.606468 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":646,"thread_start_us":407,"threads_started":1}
I20260810 11:36:19.610194 30808 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9102f493b1014764bf5dd0754073d439: Bootstrap starting.
I20260810 11:36:19.612074 8267 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 11:36:19.613932 30808 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9102f493b1014764bf5dd0754073d439: Neither blocks nor log segments found. Creating new log.
I20260810 11:36:19.615042 30808 log.cc:824] T 00000000000000000000000000000000 P 9102f493b1014764bf5dd0754073d439: Log is configured to *not* fsync() on all Append() calls
I20260810 11:36:19.618759 30808 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9102f493b1014764bf5dd0754073d439: No bootstrap required, opened a new log
I20260810 11:36:19.624482 30811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:36:19.616829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54517 (local address 127.8.18.254:41795)
0810 11:36:19.617067 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0810 11:36:19.617075 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:36:19.617563 (+ 488us) server_negotiation.cc:407] Connection header received
0810 11:36:19.617788 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:36:19.617792 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:36:19.617852 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:36:19.617936 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:36:19.618628 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:36:19.619475 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:36:19.621356 (+ 1881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:36:19.621890 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:36:19.621997 (+ 107us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:36:19.623433 (+ 1436us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:36:19.623438 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:36:19.623441 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:36:19.623463 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:36:19.623504 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:36:19.623519 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:36:19.623606 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0810 11:36:19.623878 (+ 272us) server_negotiation.cc:299] Negotiation successful
0810 11:36:19.624077 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260810 11:36:19.629173 30808 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9102f493b1014764bf5dd0754073d439 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9102f493b1014764bf5dd0754073d439" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 41795 } }
I20260810 11:36:19.629818 30808 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9102f493b1014764bf5dd0754073d439 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:36:19.629967 30808 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9102f493b1014764bf5dd0754073d439 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9102f493b1014764bf5dd0754073d439, State: Initialized, Role: FOLLOWER
I20260810 11:36:19.630703 30808 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9102f493b1014764bf5dd0754073d439 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9102f493b1014764bf5dd0754073d439" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 41795 } }
I20260810 11:36:19.630970 30808 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9102f493b1014764bf5dd0754073d439 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 11:36:19.631139 30808 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9102f493b1014764bf5dd0754073d439 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 11:36:19.631335 30808 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9102f493b1014764bf5dd0754073d439 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:36:19.633809 30808 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9102f493b1014764bf5dd0754073d439 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9102f493b1014764bf5dd0754073d439" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 41795 } }
I20260810 11:36:19.634410 30808 leader_election.cc:302] T 00000000000000000000000000000000 P 9102f493b1014764bf5dd0754073d439 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9102f493b1014764bf5dd0754073d439; no voters:
I20260810 11:36:19.635181 30808 leader_election.cc:288] T 00000000000000000000000000000000 P 9102f493b1014764bf5dd0754073d439 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 11:36:19.635318 30818 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9102f493b1014764bf5dd0754073d439 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:36:19.636406 30818 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9102f493b1014764bf5dd0754073d439 [term 1 LEADER]: Becoming Leader. State: Replica: 9102f493b1014764bf5dd0754073d439, State: Running, Role: LEADER
I20260810 11:36:19.637018 30818 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9102f493b1014764bf5dd0754073d439 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9102f493b1014764bf5dd0754073d439" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 41795 } }
I20260810 11:36:19.637625 30808 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9102f493b1014764bf5dd0754073d439 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 11:36:19.640650 30820 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9102f493b1014764bf5dd0754073d439 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9102f493b1014764bf5dd0754073d439. Latest consensus state: current_term: 1 leader_uuid: "9102f493b1014764bf5dd0754073d439" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9102f493b1014764bf5dd0754073d439" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 41795 } } }
I20260810 11:36:19.640326 30819 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9102f493b1014764bf5dd0754073d439 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9102f493b1014764bf5dd0754073d439" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9102f493b1014764bf5dd0754073d439" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 41795 } } }
I20260810 11:36:19.641093 30820 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9102f493b1014764bf5dd0754073d439 [sys.catalog]: This master's current role is: LEADER
I20260810 11:36:19.641162 30819 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9102f493b1014764bf5dd0754073d439 [sys.catalog]: This master's current role is: LEADER
I20260810 11:36:19.642381 30825 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 11:36:19.648764 30825 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 11:36:19.658342 30825 catalog_manager.cc:1347] Generated new cluster ID: c60c37740f8f4fe9b81334af21cbbd89
I20260810 11:36:19.658535 30825 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 11:36:19.680758 30825 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 11:36:19.682340 30825 catalog_manager.cc:1504] Loading token signing keys...
I20260810 11:36:19.695911 30825 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9102f493b1014764bf5dd0754073d439: Generated new TSK 0
I20260810 11:36:19.696913 30825 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 11:36:19.707209 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 30739
W20260810 11:36:19.708770 30802 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:51937
W20260810 11:36:19.709771 8267 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 30739: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 11:36:19.709952 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 30739
W20260810 11:36:19.729974 30723 connection.cc:535] client connection to 127.8.18.254:41795 recv error: Network error: failed to read from TLS socket (remote: 127.8.18.254:41795): Connection reset by peer (error 104)
2026-08-10T11:36:19Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (2089 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786361401922362-8267-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 11:36:19 dist-test-slave-srgv krb5kdc[30839](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 11:36:19 dist-test-slave-srgv krb5kdc[30839](info): set up 2 sockets
Aug 10 11:36:19 dist-test-slave-srgv krb5kdc[30839](info): commencing operation
krb5kdc: starting...
W20260810 11:36:20.889219 8267 mini_kdc.cc:121] Time spent starting KDC: real 1.140s user 0.005s sys 0.038s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786361401922362-8267-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786361401922362-8267-0/krb5kdc/test-user.keytab.
Aug 10 11:36:21 dist-test-slave-srgv krb5kdc[30839](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361781, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T11:36:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T11:36:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.18.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.18.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.18.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.254.keytab.
Entry for principal kudu/127.8.18.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.254.keytab.
I20260810 11:36:21.228854 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786361401922362-8267-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786361401922362-8267-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786361401922362-8267-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.18.254:38225
--webserver_interface=127.8.18.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.254.keytab
--principal=kudu/127.8.18.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.18.212:37123
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.18.254:38225
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786361401922362-8267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786361401922362-8267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786361401922362-8267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:36:21.601282 30854 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:36:21.601715 30854 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:36:21.601830 30854 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:36:21.609625 30854 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 11:36:21.609768 30854 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:36:21.609815 30854 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:36:21.609891 30854 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 11:36:21.609944 30854 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 11:36:21.620630 30854 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.18.212:37123
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786361401922362-8267-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786361401922362-8267-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.18.254:38225
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.254.keytab
--principal=kudu/127.8.18.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.18.254:38225
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.18.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786361401922362-8267-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:36:21.622253 30854 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:36:21.623590 30854 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:36:21.636966 30859 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:36:21.637100 30860 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:36:21.637010 30862 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:36:21.637981 30854 server_base.cc:1034] running on GCE node
I20260810 11:36:21.638974 30854 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:36:21.640584 30854 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:36:21.641844 30854 hybrid_clock.cc:648] HybridClock initialized: now 1786361781641760 us; error 72 us; skew 500 ppm
I20260810 11:36:21.642346 30854 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:36:21 dist-test-slave-srgv krb5kdc[30839](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361781, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:36:21.645924 30854 init.cc:377] Logged in from keytab as kudu/127.8.18.254@KRBTEST.COM (short username kudu)
I20260810 11:36:21.647652 30854 webserver.cc:460] Webserver started at http://127.8.18.254:38621/ using document root <none> and password file <none>
I20260810 11:36:21.648430 30854 fs_manager.cc:359] Metadata directory not provided
I20260810 11:36:21.648592 30854 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:36:21.648924 30854 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:36:21.652257 30854 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786361401922362-8267-0/minicluster-data/master-0/data/instance:
uuid: "12e80a08c96b48b0bf25f3620932c696"
format_stamp: "Formatted at 2026-08-10 11:36:21 on dist-test-slave-srgv"
server_key: "b8ebaf1ee032b8069a85e7dcf173e0ad"
server_key_iv: "d9be3d88f066b60aa7929490c5e819d5"
server_key_version: "encryptionkey@0"
I20260810 11:36:21.653172 30854 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786361401922362-8267-0/minicluster-data/master-0/wal/instance:
uuid: "12e80a08c96b48b0bf25f3620932c696"
format_stamp: "Formatted at 2026-08-10 11:36:21 on dist-test-slave-srgv"
server_key: "b8ebaf1ee032b8069a85e7dcf173e0ad"
server_key_iv: "d9be3d88f066b60aa7929490c5e819d5"
server_key_version: "encryptionkey@0"
I20260810 11:36:21.659206 30854 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260810 11:36:21.663424 30870 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:36:21.664911 30854 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 11:36:21.665139 30854 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786361401922362-8267-0/minicluster-data/master-0/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786361401922362-8267-0/minicluster-data/master-0/wal
uuid: "12e80a08c96b48b0bf25f3620932c696"
format_stamp: "Formatted at 2026-08-10 11:36:21 on dist-test-slave-srgv"
server_key: "b8ebaf1ee032b8069a85e7dcf173e0ad"
server_key_iv: "d9be3d88f066b60aa7929490c5e819d5"
server_key_version: "encryptionkey@0"
I20260810 11:36:21.665383 30854 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786361401922362-8267-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786361401922362-8267-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786361401922362-8267-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:36:21.672122 30854 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:36:21.677093 30854 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:36:21.677502 30854 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:36:21.698359 30854 rpc_server.cc:307] RPC server started. Bound to: 127.8.18.254:38225
I20260810 11:36:21.698418 30922 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.18.254:38225 every 8 connection(s)
I20260810 11:36:21.699937 30854 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
I20260810 11:36:21.703631 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 30854
I20260810 11:36:21.703863 8267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786361401922362-8267-0/minicluster-data/master-0/wal/instance
I20260810 11:36:21.704253 30923 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:36:21.704456 8267 external_mini_cluster.cc:1428] Setting key 92c18534ca18922cb0afcdf6db59ca87
I20260810 11:36:21.713276 30926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:36:21.706350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48737 (local address 127.8.18.254:38225)
0810 11:36:21.707262 (+ 912us) server_negotiation.cc:206] Beginning negotiation
0810 11:36:21.707274 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 11:36:21.707324 (+ 50us) server_negotiation.cc:407] Connection header received
0810 11:36:21.709345 (+ 2021us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:36:21.709371 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:36:21.709843 (+ 472us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:36:21.710223 (+ 380us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:36:21.711541 (+ 1318us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:36:21.711549 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:36:21.711566 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:36:21.711582 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:36:21.711659 (+ 77us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:36:21.711683 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:36:21.711874 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0810 11:36:21.711978 (+ 104us) server_negotiation.cc:299] Negotiation successful
0810 11:36:21.712269 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":661,"thread_start_us":413,"threads_started":1}
I20260810 11:36:21.717236 8267 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 11:36:21.719278 30923 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 12e80a08c96b48b0bf25f3620932c696: Bootstrap starting.
I20260810 11:36:21.723789 30923 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 12e80a08c96b48b0bf25f3620932c696: Neither blocks nor log segments found. Creating new log.
I20260810 11:36:21.725139 30923 log.cc:824] T 00000000000000000000000000000000 P 12e80a08c96b48b0bf25f3620932c696: Log is configured to *not* fsync() on all Append() calls
I20260810 11:36:21.726416 30926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:36:21.722961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42529 (local address 127.8.18.254:38225)
0810 11:36:21.723168 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0810 11:36:21.723175 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:36:21.723702 (+ 527us) server_negotiation.cc:407] Connection header received
0810 11:36:21.724233 (+ 531us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:36:21.724241 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:36:21.724325 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:36:21.724449 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:36:21.725589 (+ 1140us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:36:21.725596 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:36:21.725603 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:36:21.725621 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:36:21.725685 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:36:21.725713 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:36:21.725881 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0810 11:36:21.726033 (+ 152us) server_negotiation.cc:299] Negotiation successful
0810 11:36:21.726089 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260810 11:36:21.728937 30923 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 12e80a08c96b48b0bf25f3620932c696: No bootstrap required, opened a new log
I20260810 11:36:21.737824 30923 raft_consensus.cc:348] T 00000000000000000000000000000000 P 12e80a08c96b48b0bf25f3620932c696 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12e80a08c96b48b0bf25f3620932c696" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 38225 } }
I20260810 11:36:21.738287 30923 raft_consensus.cc:374] T 00000000000000000000000000000000 P 12e80a08c96b48b0bf25f3620932c696 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:36:21.738396 30923 raft_consensus.cc:729] T 00000000000000000000000000000000 P 12e80a08c96b48b0bf25f3620932c696 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12e80a08c96b48b0bf25f3620932c696, State: Initialized, Role: FOLLOWER
I20260810 11:36:21.739094 30923 consensus_queue.cc:260] T 00000000000000000000000000000000 P 12e80a08c96b48b0bf25f3620932c696 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12e80a08c96b48b0bf25f3620932c696" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 38225 } }
I20260810 11:36:21.739392 30923 raft_consensus.cc:388] T 00000000000000000000000000000000 P 12e80a08c96b48b0bf25f3620932c696 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 11:36:21.739555 30923 raft_consensus.cc:482] T 00000000000000000000000000000000 P 12e80a08c96b48b0bf25f3620932c696 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 11:36:21.739756 30923 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 12e80a08c96b48b0bf25f3620932c696 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:36:21.742774 30923 raft_consensus.cc:504] T 00000000000000000000000000000000 P 12e80a08c96b48b0bf25f3620932c696 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12e80a08c96b48b0bf25f3620932c696" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 38225 } }
I20260810 11:36:21.743449 30923 leader_election.cc:302] T 00000000000000000000000000000000 P 12e80a08c96b48b0bf25f3620932c696 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 12e80a08c96b48b0bf25f3620932c696; no voters:
I20260810 11:36:21.744161 30923 leader_election.cc:288] T 00000000000000000000000000000000 P 12e80a08c96b48b0bf25f3620932c696 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 11:36:21.744388 30933 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 12e80a08c96b48b0bf25f3620932c696 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:36:21.745134 30933 raft_consensus.cc:686] T 00000000000000000000000000000000 P 12e80a08c96b48b0bf25f3620932c696 [term 1 LEADER]: Becoming Leader. State: Replica: 12e80a08c96b48b0bf25f3620932c696, State: Running, Role: LEADER
I20260810 11:36:21.745779 30933 consensus_queue.cc:237] T 00000000000000000000000000000000 P 12e80a08c96b48b0bf25f3620932c696 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12e80a08c96b48b0bf25f3620932c696" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 38225 } }
I20260810 11:36:21.746528 30923 sys_catalog.cc:564] T 00000000000000000000000000000000 P 12e80a08c96b48b0bf25f3620932c696 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 11:36:21.749467 30935 sys_catalog.cc:455] T 00000000000000000000000000000000 P 12e80a08c96b48b0bf25f3620932c696 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 12e80a08c96b48b0bf25f3620932c696. Latest consensus state: current_term: 1 leader_uuid: "12e80a08c96b48b0bf25f3620932c696" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12e80a08c96b48b0bf25f3620932c696" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 38225 } } }
I20260810 11:36:21.749500 30934 sys_catalog.cc:455] T 00000000000000000000000000000000 P 12e80a08c96b48b0bf25f3620932c696 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "12e80a08c96b48b0bf25f3620932c696" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12e80a08c96b48b0bf25f3620932c696" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 38225 } } }
I20260810 11:36:21.750427 30935 sys_catalog.cc:458] T 00000000000000000000000000000000 P 12e80a08c96b48b0bf25f3620932c696 [sys.catalog]: This master's current role is: LEADER
I20260810 11:36:21.750427 30934 sys_catalog.cc:458] T 00000000000000000000000000000000 P 12e80a08c96b48b0bf25f3620932c696 [sys.catalog]: This master's current role is: LEADER
I20260810 11:36:21.751399 30941 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 11:36:21.761747 30941 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 11:36:21.773131 30941 catalog_manager.cc:1347] Generated new cluster ID: 0d31dd0b32194adcbadcd0f2ae4bd59a
I20260810 11:36:21.773296 30941 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 11:36:21.794063 30941 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 11:36:21.794795 30941 catalog_manager.cc:1504] Loading token signing keys...
I20260810 11:36:21.805469 30941 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 12e80a08c96b48b0bf25f3620932c696: Generated new TSK 0
I20260810 11:36:21.806696 30941 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 11:36:21.822819 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 30854
W20260810 11:36:21.824179 30918 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:48737
W20260810 11:36:21.824998 8267 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 30854: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 11:36:21.825173 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 30854
2026-08-10T11:36:21Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (2115 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786361401922362-8267-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 11:36:21 dist-test-slave-srgv krb5kdc[30955](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 11:36:21 dist-test-slave-srgv krb5kdc[30955](info): set up 2 sockets
Aug 10 11:36:21 dist-test-slave-srgv krb5kdc[30955](info): commencing operation
krb5kdc: starting...
W20260810 11:36:22.997179 8267 mini_kdc.cc:121] Time spent starting KDC: real 1.133s user 0.003s sys 0.043s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786361401922362-8267-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786361401922362-8267-0/krb5kdc/test-user.keytab.
Aug 10 11:36:23 dist-test-slave-srgv krb5kdc[30955](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361783, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T11:36:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T11:36:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.18.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.18.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.18.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.254.keytab.
Entry for principal kudu/127.8.18.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.254.keytab.
I20260810 11:36:23.340523 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786361401922362-8267-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786361401922362-8267-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786361401922362-8267-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.18.254:39999
--webserver_interface=127.8.18.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.254.keytab
--principal=kudu/127.8.18.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.18.212:35311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.18.254:39999
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786361401922362-8267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786361401922362-8267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786361401922362-8267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:36:23.711767 30968 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:36:23.712117 30968 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:36:23.712314 30968 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:36:23.720110 30968 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 11:36:23.720247 30968 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:36:23.720345 30968 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:36:23.720440 30968 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 11:36:23.720533 30968 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 11:36:23.731138 30968 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.18.212:35311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786361401922362-8267-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786361401922362-8267-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.18.254:39999
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.254.keytab
--principal=kudu/127.8.18.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.18.254:39999
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.18.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786361401922362-8267-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:36:23.732702 30968 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:36:23.734359 30968 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:36:23.747675 30974 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:36:23.747933 30973 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:36:23.747707 30976 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:36:23.748629 30968 server_base.cc:1034] running on GCE node
I20260810 11:36:23.749657 30968 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:36:23.751377 30968 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:36:23.752671 30968 hybrid_clock.cc:648] HybridClock initialized: now 1786361783752630 us; error 65 us; skew 500 ppm
I20260810 11:36:23.753096 30968 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:36:23 dist-test-slave-srgv krb5kdc[30955](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361783, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:36:23.760571 30968 init.cc:377] Logged in from keytab as kudu/127.8.18.254@KRBTEST.COM (short username kudu)
I20260810 11:36:23.762481 30968 webserver.cc:460] Webserver started at http://127.8.18.254:42551/ using document root <none> and password file <none>
I20260810 11:36:23.763362 30968 fs_manager.cc:359] Metadata directory not provided
I20260810 11:36:23.763537 30968 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:36:23.763877 30968 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:36:23.767319 30968 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786361401922362-8267-0/minicluster-data/master-0/data/instance:
uuid: "4b80d873c5934f93bfe539678dab514d"
format_stamp: "Formatted at 2026-08-10 11:36:23 on dist-test-slave-srgv"
server_key: "c48d0647d136f845a645ae538ce07218"
server_key_iv: "85ce886960386bbf90039238199e290f"
server_key_version: "encryptionkey@0"
I20260810 11:36:23.768208 30968 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786361401922362-8267-0/minicluster-data/master-0/wal/instance:
uuid: "4b80d873c5934f93bfe539678dab514d"
format_stamp: "Formatted at 2026-08-10 11:36:23 on dist-test-slave-srgv"
server_key: "c48d0647d136f845a645ae538ce07218"
server_key_iv: "85ce886960386bbf90039238199e290f"
server_key_version: "encryptionkey@0"
I20260810 11:36:23.774104 30968 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.004s
I20260810 11:36:23.777981 30984 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:36:23.779487 30968 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260810 11:36:23.779735 30968 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786361401922362-8267-0/minicluster-data/master-0/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786361401922362-8267-0/minicluster-data/master-0/wal
uuid: "4b80d873c5934f93bfe539678dab514d"
format_stamp: "Formatted at 2026-08-10 11:36:23 on dist-test-slave-srgv"
server_key: "c48d0647d136f845a645ae538ce07218"
server_key_iv: "85ce886960386bbf90039238199e290f"
server_key_version: "encryptionkey@0"
I20260810 11:36:23.779985 30968 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786361401922362-8267-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786361401922362-8267-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786361401922362-8267-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:36:23.794118 30968 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:36:23.799348 30968 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:36:23.799697 30968 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:36:23.820945 30968 rpc_server.cc:307] RPC server started. Bound to: 127.8.18.254:39999
I20260810 11:36:23.820990 31037 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.18.254:39999 every 8 connection(s)
I20260810 11:36:23.822630 30968 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
I20260810 11:36:23.825807 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 30968
I20260810 11:36:23.826035 8267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786361401922362-8267-0/minicluster-data/master-0/wal/instance
I20260810 11:36:23.826555 8267 external_mini_cluster.cc:1428] Setting key eea72c6dfb1cd26f8c6f8479a6ca5832
I20260810 11:36:23.827247 31038 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 10 11:36:23 dist-test-slave-srgv krb5kdc[30955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361783, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.18.254@KRBTEST.COM
I20260810 11:36:23.844288 31038 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4b80d873c5934f93bfe539678dab514d: Bootstrap starting.
I20260810 11:36:23.847998 31038 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4b80d873c5934f93bfe539678dab514d: Neither blocks nor log segments found. Creating new log.
I20260810 11:36:23.849154 31038 log.cc:824] T 00000000000000000000000000000000 P 4b80d873c5934f93bfe539678dab514d: Log is configured to *not* fsync() on all Append() calls
I20260810 11:36:23.852815 31038 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4b80d873c5934f93bfe539678dab514d: No bootstrap required, opened a new log
I20260810 11:36:23.852847 31041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:36:23.828625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47591 (local address 127.8.18.254:39999)
0810 11:36:23.829616 (+ 991us) server_negotiation.cc:206] Beginning negotiation
0810 11:36:23.829631 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 11:36:23.829679 (+ 48us) server_negotiation.cc:407] Connection header received
0810 11:36:23.831407 (+ 1728us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:36:23.831436 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:36:23.831975 (+ 539us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:36:23.832431 (+ 456us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:36:23.834675 (+ 2244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:36:23.836244 (+ 1569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:36:23.838213 (+ 1969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:36:23.838980 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:36:23.839146 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:36:23.844566 (+ 5420us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:36:23.844578 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:36:23.844597 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:36:23.844645 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:36:23.848026 (+ 3381us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:36:23.848942 (+ 916us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:36:23.848949 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:36:23.848956 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:36:23.849147 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:36:23.849638 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:36:23.849642 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:36:23.849644 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:36:23.850547 (+ 903us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:36:23.850839 (+ 292us) server_negotiation.cc:1035] Waiting for connection context
0810 11:36:23.851390 (+ 551us) server_negotiation.cc:299] Negotiation successful
0810 11:36:23.851826 (+ 436us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":716,"thread_start_us":452,"threads_started":1}
I20260810 11:36:23.857192 8267 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 11:36:23.861891 31038 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4b80d873c5934f93bfe539678dab514d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b80d873c5934f93bfe539678dab514d" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 39999 } }
I20260810 11:36:23.862448 31038 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4b80d873c5934f93bfe539678dab514d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:36:23.862589 31038 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4b80d873c5934f93bfe539678dab514d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b80d873c5934f93bfe539678dab514d, State: Initialized, Role: FOLLOWER
I20260810 11:36:23.863330 31038 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4b80d873c5934f93bfe539678dab514d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b80d873c5934f93bfe539678dab514d" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 39999 } }
I20260810 11:36:23.863649 31038 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4b80d873c5934f93bfe539678dab514d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 11:36:23.863826 31038 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4b80d873c5934f93bfe539678dab514d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 11:36:23.864058 31038 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4b80d873c5934f93bfe539678dab514d [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:36:23.867199 31038 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4b80d873c5934f93bfe539678dab514d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b80d873c5934f93bfe539678dab514d" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 39999 } }
I20260810 11:36:23.867873 31038 leader_election.cc:302] T 00000000000000000000000000000000 P 4b80d873c5934f93bfe539678dab514d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4b80d873c5934f93bfe539678dab514d; no voters:
I20260810 11:36:23.868696 31038 leader_election.cc:288] T 00000000000000000000000000000000 P 4b80d873c5934f93bfe539678dab514d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 11:36:23.868880 31048 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4b80d873c5934f93bfe539678dab514d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:36:23.871970 31038 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4b80d873c5934f93bfe539678dab514d [sys.catalog]: configured and running, proceeding with master startup.
I20260810 11:36:23.873126 31048 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4b80d873c5934f93bfe539678dab514d [term 1 LEADER]: Becoming Leader. State: Replica: 4b80d873c5934f93bfe539678dab514d, State: Running, Role: LEADER
I20260810 11:36:23.873994 31048 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4b80d873c5934f93bfe539678dab514d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b80d873c5934f93bfe539678dab514d" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 39999 } }
I20260810 11:36:23.878185 31050 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4b80d873c5934f93bfe539678dab514d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4b80d873c5934f93bfe539678dab514d. Latest consensus state: current_term: 1 leader_uuid: "4b80d873c5934f93bfe539678dab514d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b80d873c5934f93bfe539678dab514d" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 39999 } } }
I20260810 11:36:23.878074 31049 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4b80d873c5934f93bfe539678dab514d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4b80d873c5934f93bfe539678dab514d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b80d873c5934f93bfe539678dab514d" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 39999 } } }
I20260810 11:36:23.878564 31049 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4b80d873c5934f93bfe539678dab514d [sys.catalog]: This master's current role is: LEADER
I20260810 11:36:23.878564 31050 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4b80d873c5934f93bfe539678dab514d [sys.catalog]: This master's current role is: LEADER
I20260810 11:36:23.879660 31059 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 11:36:23.883665 31041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:36:23.862208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42477 (local address 127.8.18.254:39999)
0810 11:36:23.862423 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0810 11:36:23.862435 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 11:36:23.863043 (+ 608us) server_negotiation.cc:407] Connection header received
0810 11:36:23.863275 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:36:23.863281 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:36:23.863362 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:36:23.863450 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:36:23.865684 (+ 2234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:36:23.866490 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:36:23.868422 (+ 1932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:36:23.869356 (+ 934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:36:23.869607 (+ 251us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:36:23.872842 (+ 3235us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:36:23.872849 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:36:23.872855 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:36:23.872897 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:36:23.877659 (+ 4762us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:36:23.879013 (+ 1354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:36:23.879021 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:36:23.879027 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:36:23.879353 (+ 326us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:36:23.880010 (+ 657us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:36:23.880018 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:36:23.880024 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:36:23.880716 (+ 692us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:36:23.882560 (+ 1844us) server_negotiation.cc:1035] Waiting for connection context
0810 11:36:23.882771 (+ 211us) server_negotiation.cc:299] Negotiation successful
0810 11:36:23.883117 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260810 11:36:23.894822 31059 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 11:36:23.905436 31059 catalog_manager.cc:1347] Generated new cluster ID: 292fb731b4e4422383d12ed9f2d9a521
I20260810 11:36:23.905596 31059 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 11:36:23.922837 31059 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 11:36:23.925083 31059 catalog_manager.cc:1504] Loading token signing keys...
I20260810 11:36:23.933482 31059 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4b80d873c5934f93bfe539678dab514d: Generated new TSK 0
I20260810 11:36:23.934733 31059 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 11:36:23.941783 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 30968
I20260810 11:36:23.943140 31033 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:36:24.042958 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 30968
2026-08-10T11:36:24Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (2218 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786361401922362-8267-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 11:36:24 dist-test-slave-srgv krb5kdc[31071](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 11:36:24 dist-test-slave-srgv krb5kdc[31071](info): set up 2 sockets
Aug 10 11:36:24 dist-test-slave-srgv krb5kdc[31071](info): commencing operation
krb5kdc: starting...
W20260810 11:36:25.196910 8267 mini_kdc.cc:121] Time spent starting KDC: real 1.114s user 0.000s sys 0.044s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786361401922362-8267-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786361401922362-8267-0/krb5kdc/test-user.keytab.
Aug 10 11:36:25 dist-test-slave-srgv krb5kdc[31071](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361785, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T11:36:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T11:36:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.18.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.18.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.18.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.254.keytab.
Entry for principal kudu/127.8.18.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.254.keytab.
I20260810 11:36:25.549496 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786361401922362-8267-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786361401922362-8267-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786361401922362-8267-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.18.254:42041
--webserver_interface=127.8.18.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.254.keytab
--principal=kudu/127.8.18.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.18.212:37331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.18.254:42041
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786361401922362-8267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786361401922362-8267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786361401922362-8267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:36:25.915067 31086 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:36:25.915395 31086 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:36:25.915494 31086 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:36:25.923085 31086 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 11:36:25.923226 31086 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:36:25.923275 31086 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:36:25.923332 31086 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 11:36:25.923380 31086 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 11:36:25.933864 31086 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.18.212:37331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786361401922362-8267-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786361401922362-8267-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.18.254:42041
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.254.keytab
--principal=kudu/127.8.18.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.18.254:42041
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.18.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786361401922362-8267-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:36:25.935230 31086 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:36:25.936596 31086 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:36:25.951581 31091 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:36:25.951915 31092 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:36:25.952107 31094 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:36:25.952805 31086 server_base.cc:1034] running on GCE node
I20260810 11:36:25.953769 31086 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:36:25.955329 31086 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:36:25.956579 31086 hybrid_clock.cc:648] HybridClock initialized: now 1786361785956515 us; error 58 us; skew 500 ppm
I20260810 11:36:25.956964 31086 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:36:25 dist-test-slave-srgv krb5kdc[31071](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361785, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:36:25.960634 31086 init.cc:377] Logged in from keytab as kudu/127.8.18.254@KRBTEST.COM (short username kudu)
I20260810 11:36:25.962451 31086 webserver.cc:460] Webserver started at http://127.8.18.254:38101/ using document root <none> and password file <none>
I20260810 11:36:25.963232 31086 fs_manager.cc:359] Metadata directory not provided
I20260810 11:36:25.963395 31086 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:36:25.963745 31086 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:36:25.967111 31086 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786361401922362-8267-0/minicluster-data/master-0/data/instance:
uuid: "bddaba56d82c4093a7cf40edc2d7bf26"
format_stamp: "Formatted at 2026-08-10 11:36:25 on dist-test-slave-srgv"
server_key: "c8819df47f894a41d0f8b02be770eab8"
server_key_iv: "2674feb90ed89b6b158cc1472402a1a5"
server_key_version: "encryptionkey@0"
I20260810 11:36:25.967972 31086 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786361401922362-8267-0/minicluster-data/master-0/wal/instance:
uuid: "bddaba56d82c4093a7cf40edc2d7bf26"
format_stamp: "Formatted at 2026-08-10 11:36:25 on dist-test-slave-srgv"
server_key: "c8819df47f894a41d0f8b02be770eab8"
server_key_iv: "2674feb90ed89b6b158cc1472402a1a5"
server_key_version: "encryptionkey@0"
I20260810 11:36:25.974097 31086 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.003s
I20260810 11:36:25.977958 31102 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:36:25.979414 31086 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 11:36:25.979624 31086 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786361401922362-8267-0/minicluster-data/master-0/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786361401922362-8267-0/minicluster-data/master-0/wal
uuid: "bddaba56d82c4093a7cf40edc2d7bf26"
format_stamp: "Formatted at 2026-08-10 11:36:25 on dist-test-slave-srgv"
server_key: "c8819df47f894a41d0f8b02be770eab8"
server_key_iv: "2674feb90ed89b6b158cc1472402a1a5"
server_key_version: "encryptionkey@0"
I20260810 11:36:25.979861 31086 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786361401922362-8267-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786361401922362-8267-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786361401922362-8267-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:36:25.999971 31086 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:36:26.004702 31086 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:36:26.005020 31086 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:36:26.025934 31086 rpc_server.cc:307] RPC server started. Bound to: 127.8.18.254:42041
I20260810 11:36:26.025976 31155 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.18.254:42041 every 8 connection(s)
I20260810 11:36:26.027467 31086 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
I20260810 11:36:26.031570 31156 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:36:26.033061 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 31086
I20260810 11:36:26.033277 8267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786361401922362-8267-0/minicluster-data/master-0/wal/instance
I20260810 11:36:26.033772 8267 external_mini_cluster.cc:1428] Setting key e2abb7de55a3606bfad29a01cd5ac092
I20260810 11:36:26.046571 31156 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bddaba56d82c4093a7cf40edc2d7bf26: Bootstrap starting.
I20260810 11:36:26.047945 31159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:36:26.035621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43719 (local address 127.8.18.254:42041)
0810 11:36:26.036499 (+ 878us) server_negotiation.cc:206] Beginning negotiation
0810 11:36:26.036511 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 11:36:26.036559 (+ 48us) server_negotiation.cc:407] Connection header received
0810 11:36:26.038341 (+ 1782us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:36:26.038372 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:36:26.038814 (+ 442us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:36:26.039173 (+ 359us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:36:26.039856 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:36:26.041152 (+ 1296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:36:26.043135 (+ 1983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:36:26.043937 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:36:26.044094 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:36:26.045569 (+ 1475us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:36:26.045578 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:36:26.045594 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:36:26.046180 (+ 586us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:36:26.046271 (+ 91us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:36:26.046297 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:36:26.046389 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0810 11:36:26.046642 (+ 253us) server_negotiation.cc:299] Negotiation successful
0810 11:36:26.046936 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":649,"thread_start_us":423,"threads_started":1}
I20260810 11:36:26.050523 31156 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bddaba56d82c4093a7cf40edc2d7bf26: Neither blocks nor log segments found. Creating new log.
I20260810 11:36:26.051913 31156 log.cc:824] T 00000000000000000000000000000000 P bddaba56d82c4093a7cf40edc2d7bf26: Log is configured to *not* fsync() on all Append() calls
I20260810 11:36:26.052287 8267 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 11:36:26.055130 31156 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bddaba56d82c4093a7cf40edc2d7bf26: No bootstrap required, opened a new log
I20260810 11:36:26.063720 31159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:36:26.057310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42225 (local address 127.8.18.254:42041)
0810 11:36:26.057540 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0810 11:36:26.057549 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 11:36:26.057951 (+ 402us) server_negotiation.cc:407] Connection header received
0810 11:36:26.058184 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:36:26.058188 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:36:26.058255 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:36:26.058382 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:36:26.058951 (+ 569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:36:26.059682 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:36:26.061079 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:36:26.061623 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:36:26.061797 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:36:26.062956 (+ 1159us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:36:26.062961 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:36:26.062964 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:36:26.062985 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:36:26.063027 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:36:26.063043 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:36:26.063141 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0810 11:36:26.063301 (+ 160us) server_negotiation.cc:299] Negotiation successful
0810 11:36:26.063400 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260810 11:36:26.063730 31156 raft_consensus.cc:348] T 00000000000000000000000000000000 P bddaba56d82c4093a7cf40edc2d7bf26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bddaba56d82c4093a7cf40edc2d7bf26" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 42041 } }
I20260810 11:36:26.064409 31156 raft_consensus.cc:374] T 00000000000000000000000000000000 P bddaba56d82c4093a7cf40edc2d7bf26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:36:26.064561 31156 raft_consensus.cc:729] T 00000000000000000000000000000000 P bddaba56d82c4093a7cf40edc2d7bf26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bddaba56d82c4093a7cf40edc2d7bf26, State: Initialized, Role: FOLLOWER
I20260810 11:36:26.065274 31156 consensus_queue.cc:260] T 00000000000000000000000000000000 P bddaba56d82c4093a7cf40edc2d7bf26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bddaba56d82c4093a7cf40edc2d7bf26" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 42041 } }
I20260810 11:36:26.065653 31156 raft_consensus.cc:388] T 00000000000000000000000000000000 P bddaba56d82c4093a7cf40edc2d7bf26 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 11:36:26.065819 31156 raft_consensus.cc:482] T 00000000000000000000000000000000 P bddaba56d82c4093a7cf40edc2d7bf26 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 11:36:26.065960 31156 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bddaba56d82c4093a7cf40edc2d7bf26 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:36:26.068576 31156 raft_consensus.cc:504] T 00000000000000000000000000000000 P bddaba56d82c4093a7cf40edc2d7bf26 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bddaba56d82c4093a7cf40edc2d7bf26" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 42041 } }
I20260810 11:36:26.069306 31156 leader_election.cc:302] T 00000000000000000000000000000000 P bddaba56d82c4093a7cf40edc2d7bf26 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bddaba56d82c4093a7cf40edc2d7bf26; no voters:
I20260810 11:36:26.070000 31156 leader_election.cc:288] T 00000000000000000000000000000000 P bddaba56d82c4093a7cf40edc2d7bf26 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 11:36:26.070161 31166 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bddaba56d82c4093a7cf40edc2d7bf26 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:36:26.070961 31166 raft_consensus.cc:686] T 00000000000000000000000000000000 P bddaba56d82c4093a7cf40edc2d7bf26 [term 1 LEADER]: Becoming Leader. State: Replica: bddaba56d82c4093a7cf40edc2d7bf26, State: Running, Role: LEADER
I20260810 11:36:26.071601 31166 consensus_queue.cc:237] T 00000000000000000000000000000000 P bddaba56d82c4093a7cf40edc2d7bf26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bddaba56d82c4093a7cf40edc2d7bf26" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 42041 } }
I20260810 11:36:26.072373 31156 sys_catalog.cc:564] T 00000000000000000000000000000000 P bddaba56d82c4093a7cf40edc2d7bf26 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 11:36:26.075192 31167 sys_catalog.cc:455] T 00000000000000000000000000000000 P bddaba56d82c4093a7cf40edc2d7bf26 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bddaba56d82c4093a7cf40edc2d7bf26" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bddaba56d82c4093a7cf40edc2d7bf26" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 42041 } } }
I20260810 11:36:26.075204 31168 sys_catalog.cc:455] T 00000000000000000000000000000000 P bddaba56d82c4093a7cf40edc2d7bf26 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bddaba56d82c4093a7cf40edc2d7bf26. Latest consensus state: current_term: 1 leader_uuid: "bddaba56d82c4093a7cf40edc2d7bf26" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bddaba56d82c4093a7cf40edc2d7bf26" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 42041 } } }
I20260810 11:36:26.075742 31167 sys_catalog.cc:458] T 00000000000000000000000000000000 P bddaba56d82c4093a7cf40edc2d7bf26 [sys.catalog]: This master's current role is: LEADER
I20260810 11:36:26.075809 31168 sys_catalog.cc:458] T 00000000000000000000000000000000 P bddaba56d82c4093a7cf40edc2d7bf26 [sys.catalog]: This master's current role is: LEADER
I20260810 11:36:26.076743 31172 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 11:36:26.082791 31172 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 11:36:26.093546 31172 catalog_manager.cc:1347] Generated new cluster ID: 7c2ce660e96d47fd8487c196d07ad453
I20260810 11:36:26.093680 31172 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 11:36:26.115788 31172 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 11:36:26.117854 31172 catalog_manager.cc:1504] Loading token signing keys...
I20260810 11:36:26.134716 31172 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bddaba56d82c4093a7cf40edc2d7bf26: Generated new TSK 0
I20260810 11:36:26.135939 31172 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 11:36:26.149776 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 31086
W20260810 11:36:26.151064 31151 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:43719
W20260810 11:36:26.151875 8267 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 31086: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 11:36:26.152041 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 31086
2026-08-10T11:36:26Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (2108 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786361401922362-8267-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 11:36:26 dist-test-slave-srgv krb5kdc[31187](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 11:36:26 dist-test-slave-srgv krb5kdc[31187](info): set up 2 sockets
Aug 10 11:36:26 dist-test-slave-srgv krb5kdc[31187](info): commencing operation
krb5kdc: starting...
W20260810 11:36:27.434084 8267 mini_kdc.cc:121] Time spent starting KDC: real 1.243s user 0.000s sys 0.044s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786361401922362-8267-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786361401922362-8267-0/krb5kdc/test-user.keytab.
Aug 10 11:36:27 dist-test-slave-srgv krb5kdc[31187](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361787, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T11:36:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T11:36:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.18.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.18.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.18.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.254.keytab.
Entry for principal kudu/127.8.18.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.254.keytab.
W20260810 11:36:27.988516 8267 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.8.18.254: real 0.113s user 0.000s sys 0.042s
I20260810 11:36:27.989967 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786361401922362-8267-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786361401922362-8267-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786361401922362-8267-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.18.254:39943
--webserver_interface=127.8.18.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.254.keytab
--principal=kudu/127.8.18.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.18.212:32927
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.18.254:39943
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786361401922362-8267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786361401922362-8267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786361401922362-8267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:36:28.619946 31200 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:36:28.620296 31200 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:36:28.620510 31200 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:36:28.629505 31200 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 11:36:28.629670 31200 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:36:28.629781 31200 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:36:28.629876 31200 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 11:36:28.629951 31200 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 11:36:28.641255 31200 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.18.212:32927
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786361401922362-8267-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786361401922362-8267-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.18.254:39943
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.254.keytab
--principal=kudu/127.8.18.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.18.254:39943
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.18.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786361401922362-8267-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:36:28.642798 31200 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:36:28.644240 31200 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:36:28.657541 31206 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:36:28.657514 31210 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:36:28.657481 31208 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:36:28.658793 31200 server_base.cc:1034] running on GCE node
I20260810 11:36:28.659688 31200 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:36:28.661283 31200 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:36:28.662571 31200 hybrid_clock.cc:648] HybridClock initialized: now 1786361788662532 us; error 82 us; skew 500 ppm
I20260810 11:36:28.663017 31200 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:36:28 dist-test-slave-srgv krb5kdc[31187](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361788, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:36:28.666615 31200 init.cc:377] Logged in from keytab as kudu/127.8.18.254@KRBTEST.COM (short username kudu)
I20260810 11:36:28.668501 31200 webserver.cc:460] Webserver started at http://127.8.18.254:44653/ using document root <none> and password file <none>
I20260810 11:36:28.669270 31200 fs_manager.cc:359] Metadata directory not provided
I20260810 11:36:28.669487 31200 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:36:28.669835 31200 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:36:28.673110 31200 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786361401922362-8267-0/minicluster-data/master-0/data/instance:
uuid: "53e74092dfad433e92434220a414342a"
format_stamp: "Formatted at 2026-08-10 11:36:28 on dist-test-slave-srgv"
server_key: "97ffbf0e42918ff296840e9b4545b62b"
server_key_iv: "39e6a685334dd6c6b1c582f0ac3b11a2"
server_key_version: "encryptionkey@0"
I20260810 11:36:28.674043 31200 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786361401922362-8267-0/minicluster-data/master-0/wal/instance:
uuid: "53e74092dfad433e92434220a414342a"
format_stamp: "Formatted at 2026-08-10 11:36:28 on dist-test-slave-srgv"
server_key: "97ffbf0e42918ff296840e9b4545b62b"
server_key_iv: "39e6a685334dd6c6b1c582f0ac3b11a2"
server_key_version: "encryptionkey@0"
I20260810 11:36:28.680014 31200 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.001s sys 0.004s
I20260810 11:36:28.684135 31217 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:36:28.685727 31200 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260810 11:36:28.685932 31200 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786361401922362-8267-0/minicluster-data/master-0/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786361401922362-8267-0/minicluster-data/master-0/wal
uuid: "53e74092dfad433e92434220a414342a"
format_stamp: "Formatted at 2026-08-10 11:36:28 on dist-test-slave-srgv"
server_key: "97ffbf0e42918ff296840e9b4545b62b"
server_key_iv: "39e6a685334dd6c6b1c582f0ac3b11a2"
server_key_version: "encryptionkey@0"
I20260810 11:36:28.686170 31200 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786361401922362-8267-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786361401922362-8267-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786361401922362-8267-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:36:28.692698 31200 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:36:28.697319 31200 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:36:28.697737 31200 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:36:28.718880 31200 rpc_server.cc:307] RPC server started. Bound to: 127.8.18.254:39943
I20260810 11:36:28.719065 31269 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.18.254:39943 every 8 connection(s)
I20260810 11:36:28.720494 31200 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
I20260810 11:36:28.724505 31270 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:36:28.729579 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 31200
I20260810 11:36:28.729804 8267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786361401922362-8267-0/minicluster-data/master-0/wal/instance
I20260810 11:36:28.730412 8267 external_mini_cluster.cc:1428] Setting key bdd5952468bba5d8bcae24b16f6f9c01
I20260810 11:36:28.737028 31270 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 53e74092dfad433e92434220a414342a: Bootstrap starting.
I20260810 11:36:28.739957 31273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:36:28.732437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60571 (local address 127.8.18.254:39943)
0810 11:36:28.733464 (+ 1027us) server_negotiation.cc:206] Beginning negotiation
0810 11:36:28.733483 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0810 11:36:28.733568 (+ 85us) server_negotiation.cc:407] Connection header received
0810 11:36:28.735787 (+ 2219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:36:28.735817 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:36:28.736228 (+ 411us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:36:28.736721 (+ 493us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:36:28.738042 (+ 1321us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:36:28.738051 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:36:28.738068 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:36:28.738083 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:36:28.738151 (+ 68us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:36:28.738185 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:36:28.738468 (+ 283us) server_negotiation.cc:1035] Waiting for connection context
0810 11:36:28.738680 (+ 212us) server_negotiation.cc:299] Negotiation successful
0810 11:36:28.738900 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":740,"thread_start_us":454,"threads_started":1}
I20260810 11:36:28.740901 31270 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 53e74092dfad433e92434220a414342a: Neither blocks nor log segments found. Creating new log.
I20260810 11:36:28.742264 31270 log.cc:824] T 00000000000000000000000000000000 P 53e74092dfad433e92434220a414342a: Log is configured to *not* fsync() on all Append() calls
I20260810 11:36:28.744876 8267 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 11:36:28.746290 31270 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 53e74092dfad433e92434220a414342a: No bootstrap required, opened a new log
I20260810 11:36:28.753633 31273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:36:28.750612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34699 (local address 127.8.18.254:39943)
0810 11:36:28.750886 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0810 11:36:28.750897 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 11:36:28.751230 (+ 333us) server_negotiation.cc:407] Connection header received
0810 11:36:28.751465 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:36:28.751469 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:36:28.751540 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:36:28.751670 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:36:28.752813 (+ 1143us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:36:28.752821 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:36:28.752824 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 11:36:28.752847 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:36:28.752894 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 11:36:28.752913 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:36:28.753068 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0810 11:36:28.753186 (+ 118us) server_negotiation.cc:299] Negotiation successful
0810 11:36:28.753271 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113}
I20260810 11:36:28.756717 31270 raft_consensus.cc:348] T 00000000000000000000000000000000 P 53e74092dfad433e92434220a414342a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53e74092dfad433e92434220a414342a" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 39943 } }
I20260810 11:36:28.757352 31270 raft_consensus.cc:374] T 00000000000000000000000000000000 P 53e74092dfad433e92434220a414342a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 11:36:28.757516 31270 raft_consensus.cc:729] T 00000000000000000000000000000000 P 53e74092dfad433e92434220a414342a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53e74092dfad433e92434220a414342a, State: Initialized, Role: FOLLOWER
I20260810 11:36:28.758312 31270 consensus_queue.cc:260] T 00000000000000000000000000000000 P 53e74092dfad433e92434220a414342a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53e74092dfad433e92434220a414342a" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 39943 } }
I20260810 11:36:28.758567 31270 raft_consensus.cc:388] T 00000000000000000000000000000000 P 53e74092dfad433e92434220a414342a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 11:36:28.758671 31270 raft_consensus.cc:482] T 00000000000000000000000000000000 P 53e74092dfad433e92434220a414342a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 11:36:28.758811 31270 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 53e74092dfad433e92434220a414342a [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:36:28.761523 31270 raft_consensus.cc:504] T 00000000000000000000000000000000 P 53e74092dfad433e92434220a414342a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53e74092dfad433e92434220a414342a" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 39943 } }
I20260810 11:36:28.762193 31270 leader_election.cc:302] T 00000000000000000000000000000000 P 53e74092dfad433e92434220a414342a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 53e74092dfad433e92434220a414342a; no voters:
I20260810 11:36:28.762984 31270 leader_election.cc:288] T 00000000000000000000000000000000 P 53e74092dfad433e92434220a414342a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 11:36:28.763121 31280 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 53e74092dfad433e92434220a414342a [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:36:28.763857 31280 raft_consensus.cc:686] T 00000000000000000000000000000000 P 53e74092dfad433e92434220a414342a [term 1 LEADER]: Becoming Leader. State: Replica: 53e74092dfad433e92434220a414342a, State: Running, Role: LEADER
I20260810 11:36:28.764663 31280 consensus_queue.cc:237] T 00000000000000000000000000000000 P 53e74092dfad433e92434220a414342a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53e74092dfad433e92434220a414342a" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 39943 } }
I20260810 11:36:28.765614 31270 sys_catalog.cc:564] T 00000000000000000000000000000000 P 53e74092dfad433e92434220a414342a [sys.catalog]: configured and running, proceeding with master startup.
I20260810 11:36:28.768159 31281 sys_catalog.cc:455] T 00000000000000000000000000000000 P 53e74092dfad433e92434220a414342a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "53e74092dfad433e92434220a414342a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53e74092dfad433e92434220a414342a" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 39943 } } }
I20260810 11:36:28.768102 31282 sys_catalog.cc:455] T 00000000000000000000000000000000 P 53e74092dfad433e92434220a414342a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 53e74092dfad433e92434220a414342a. Latest consensus state: current_term: 1 leader_uuid: "53e74092dfad433e92434220a414342a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53e74092dfad433e92434220a414342a" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 39943 } } }
I20260810 11:36:28.768719 31281 sys_catalog.cc:458] T 00000000000000000000000000000000 P 53e74092dfad433e92434220a414342a [sys.catalog]: This master's current role is: LEADER
I20260810 11:36:28.768785 31282 sys_catalog.cc:458] T 00000000000000000000000000000000 P 53e74092dfad433e92434220a414342a [sys.catalog]: This master's current role is: LEADER
I20260810 11:36:28.771610 31288 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 11:36:28.777597 31288 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 11:36:28.786597 31288 catalog_manager.cc:1347] Generated new cluster ID: 4b8499d09646401490bb2b3b53ad3336
I20260810 11:36:28.786733 31288 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 11:36:28.814378 31288 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 11:36:28.814951 31288 catalog_manager.cc:1504] Loading token signing keys...
I20260810 11:36:28.827414 31288 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 53e74092dfad433e92434220a414342a: Generated new TSK 0
I20260810 11:36:28.828436 31288 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 11:36:28.853495 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 31200
W20260810 11:36:28.854872 31265 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:60571
W20260810 11:36:28.855674 8267 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 31200: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 11:36:28.855835 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 31200
2026-08-10T11:36:28Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (2704 ms)
[----------] 18 tests from AuthTokenIssuingTest (39339 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786361401922362-8267-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 11:36:28 dist-test-slave-srgv krb5kdc[31302](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 11:36:28 dist-test-slave-srgv krb5kdc[31302](info): set up 2 sockets
Aug 10 11:36:28 dist-test-slave-srgv krb5kdc[31302](info): commencing operation
krb5kdc: starting...
W20260810 11:36:30.013638 8267 mini_kdc.cc:121] Time spent starting KDC: real 1.115s user 0.007s sys 0.038s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786361401922362-8267-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786361401922362-8267-0/krb5kdc/test-user.keytab.
Aug 10 11:36:30 dist-test-slave-srgv krb5kdc[31302](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361790, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T11:36:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T11:36:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.18.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.18.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.18.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.254.keytab.
Entry for principal kudu/127.8.18.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.254.keytab.
I20260810 11:36:30.357177 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786361401922362-8267-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786361401922362-8267-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786361401922362-8267-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.18.254:34903
--webserver_interface=127.8.18.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.254.keytab
--principal=kudu/127.8.18.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.18.212:40305
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.18.254:34903,127.8.18.253:41297,127.8.18.252:40443
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786361401922362-8267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786361401922362-8267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786361401922362-8267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:36:30.725047 31315 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260810 11:36:30.725509 31315 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:36:30.725648 31315 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:36:30.725785 31315 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:36:30.733465 31315 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 11:36:30.733623 31315 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:36:30.733729 31315 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:36:30.733819 31315 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 11:36:30.733929 31315 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 11:36:30.744410 31315 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.18.212:40305
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786361401922362-8267-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786361401922362-8267-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.18.254:34903,127.8.18.253:41297,127.8.18.252:40443
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.254.keytab
--principal=kudu/127.8.18.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.18.254:34903
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.18.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786361401922362-8267-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:36:30.745946 31315 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:36:30.747360 31315 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:36:30.759914 31321 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:36:30.759919 31322 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:36:30.760030 31324 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:36:30.761194 31315 server_base.cc:1034] running on GCE node
I20260810 11:36:30.762135 31315 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:36:30.763634 31315 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:36:30.764869 31315 hybrid_clock.cc:648] HybridClock initialized: now 1786361790764808 us; error 33 us; skew 500 ppm
I20260810 11:36:30.765244 31315 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:36:30 dist-test-slave-srgv krb5kdc[31302](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361790, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:36:30.768705 31315 init.cc:377] Logged in from keytab as kudu/127.8.18.254@KRBTEST.COM (short username kudu)
I20260810 11:36:30.770484 31315 webserver.cc:460] Webserver started at http://127.8.18.254:40145/ using document root <none> and password file <none>
I20260810 11:36:30.771241 31315 fs_manager.cc:359] Metadata directory not provided
I20260810 11:36:30.771384 31315 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:36:30.771739 31315 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:36:30.775022 31315 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786361401922362-8267-0/minicluster-data/master-0/data/instance:
uuid: "06ec21c7adf84c24a54cbaa5cd53c462"
format_stamp: "Formatted at 2026-08-10 11:36:30 on dist-test-slave-srgv"
server_key: "47706b997ec90df96bb0e389a70bdd14"
server_key_iv: "c48f2c44ad41603ec982f19aca293068"
server_key_version: "encryptionkey@0"
I20260810 11:36:30.775913 31315 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786361401922362-8267-0/minicluster-data/master-0/wal/instance:
uuid: "06ec21c7adf84c24a54cbaa5cd53c462"
format_stamp: "Formatted at 2026-08-10 11:36:30 on dist-test-slave-srgv"
server_key: "47706b997ec90df96bb0e389a70bdd14"
server_key_iv: "c48f2c44ad41603ec982f19aca293068"
server_key_version: "encryptionkey@0"
I20260810 11:36:30.781843 31315 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.005s
I20260810 11:36:30.785679 31331 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:36:30.787218 31315 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 11:36:30.787446 31315 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786361401922362-8267-0/minicluster-data/master-0/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786361401922362-8267-0/minicluster-data/master-0/wal
uuid: "06ec21c7adf84c24a54cbaa5cd53c462"
format_stamp: "Formatted at 2026-08-10 11:36:30 on dist-test-slave-srgv"
server_key: "47706b997ec90df96bb0e389a70bdd14"
server_key_iv: "c48f2c44ad41603ec982f19aca293068"
server_key_version: "encryptionkey@0"
I20260810 11:36:30.787696 31315 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786361401922362-8267-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786361401922362-8267-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786361401922362-8267-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:36:30.814908 31315 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:36:30.819775 31315 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:36:30.820147 31315 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:36:30.840821 31315 rpc_server.cc:307] RPC server started. Bound to: 127.8.18.254:34903
I20260810 11:36:30.840827 31384 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.18.254:34903 every 8 connection(s)
I20260810 11:36:30.842357 31315 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
I20260810 11:36:30.846271 31385 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:36:30.850972 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 31315
I20260810 11:36:30.851217 8267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786361401922362-8267-0/minicluster-data/master-0/wal/instance
I20260810 11:36:30.851883 8267 external_mini_cluster.cc:1428] Setting key 6d5a41b354e327d3419ac9a38d21f73e
I20260810 11:36:30.853204 31385 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.18.254" port: 34903 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 11:36:30 dist-test-slave-srgv krb5kdc[31302](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361790, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.254@KRBTEST.COM for kudu/127.8.18.254@KRBTEST.COM
I20260810 11:36:30.877425 31388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:36:30.856642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:34903 (local address 127.0.0.1:35018)
0810 11:36:30.858001 (+ 1359us) negotiation.cc:107] Waiting for socket to connect
0810 11:36:30.858043 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 11:36:30.859931 (+ 1888us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:36:30.860846 (+ 915us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:36:30.860853 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:36:30.861665 (+ 812us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:36:30.862028 (+ 363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:36:30.862045 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:36:30.863697 (+ 1652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:36:30.863702 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:36:30.864846 (+ 1144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:36:30.864858 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:36:30.865952 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:36:30.865958 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:36:30.866130 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:36:30.866893 (+ 763us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:36:30.866918 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:36:30.869321 (+ 2403us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:36:30.872769 (+ 3448us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:36:30.872782 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:36:30.872796 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:36:30.873205 (+ 409us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:36:30.873719 (+ 514us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:36:30.873733 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:36:30.873736 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:36:30.873916 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:36:30.874759 (+ 843us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:36:30.874767 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:36:30.874982 (+ 215us) client_negotiation.cc:769] Sending connection context
0810 11:36:30.875235 (+ 253us) client_negotiation.cc:240] Negotiation successful
0810 11:36:30.875940 (+ 705us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1153,"mutex_wait_us":200,"thread_start_us":871,"threads_started":1}
I20260810 11:36:30.877451 31387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:36:30.856641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35018 (local address 127.8.18.254:34903)
0810 11:36:30.857696 (+ 1055us) server_negotiation.cc:206] Beginning negotiation
0810 11:36:30.857717 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0810 11:36:30.858177 (+ 460us) server_negotiation.cc:407] Connection header received
0810 11:36:30.860347 (+ 2170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:36:30.860353 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:36:30.860601 (+ 248us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:36:30.860820 (+ 219us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:36:30.862265 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:36:30.863515 (+ 1250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:36:30.865001 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:36:30.865692 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:36:30.865910 (+ 218us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:36:30.869638 (+ 3728us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:36:30.869647 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:36:30.869662 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:36:30.869694 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:36:30.872611 (+ 2917us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:36:30.873362 (+ 751us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:36:30.873369 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:36:30.873372 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:36:30.873598 (+ 226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:36:30.874042 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:36:30.874053 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:36:30.874056 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:36:30.874600 (+ 544us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:36:30.874832 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0810 11:36:30.875387 (+ 555us) server_negotiation.cc:299] Negotiation successful
0810 11:36:30.875810 (+ 423us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":48,"server-negotiator.queue_time_us":785,"thread_start_us":482,"threads_started":1}
I20260810 11:36:30.880860 31385 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.18.253" port: 41297 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.8.18.253@KRBTEST.COM; defaulting to no policy
I20260810 11:36:30.882174 31388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:36:30.881692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.253:41297 (local address 127.0.0.1:33410)
0810 11:36:30.881870 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0810 11:36:30.882033 (+ 163us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.253:41297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260810 11:36:30.882534 31332 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.18.253:41297: connect: Connection refused (error 111)
I20260810 11:36:30.884210 31388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:36:30.883804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.253:41297 (local address 127.0.0.1:33412)
0810 11:36:30.883960 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0810 11:36:30.884053 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.253:41297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260810 11:36:30.884501 31385 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.18.253:41297: Network error: Client connection negotiation failed: client connection to 127.8.18.253:41297: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.18.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.18.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.253.keytab.
Entry for principal kudu/127.8.18.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.253.keytab.
I20260810 11:36:30.923173 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786361401922362-8267-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786361401922362-8267-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786361401922362-8267-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786361401922362-8267-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.18.253:41297
--webserver_interface=127.8.18.253
--webserver_port=0
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.253.keytab
--principal=kudu/127.8.18.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.18.212:40305
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.18.254:34903,127.8.18.253:41297,127.8.18.252:40443
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786361401922362-8267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786361401922362-8267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786361401922362-8267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 11:36:30.933940 31385 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.18.253" port: 41297 } attempt: 1
I20260810 11:36:30.935326 31388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:36:30.934822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.253:41297 (local address 127.0.0.1:33416)
0810 11:36:30.935057 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0810 11:36:30.935186 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.253:41297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260810 11:36:30.936568 31388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:36:30.936128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.253:41297 (local address 127.0.0.1:33432)
0810 11:36:30.936299 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0810 11:36:30.936372 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.253:41297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260810 11:36:30.936779 31385 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.18.253:41297: Network error: Client connection negotiation failed: client connection to 127.8.18.253:41297: connect: Connection refused (error 111)
I20260810 11:36:31.005196 31385 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.18.253" port: 41297 } attempt: 2
I20260810 11:36:31.006515 31388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:36:31.006008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.253:41297 (local address 127.0.0.1:33436)
0810 11:36:31.006232 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0810 11:36:31.006342 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.253:41297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260810 11:36:31.007635 31388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:36:31.007230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.253:41297 (local address 127.0.0.1:33446)
0810 11:36:31.007388 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0810 11:36:31.007481 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.253:41297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260810 11:36:31.008044 31385 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.18.253:41297: Network error: Client connection negotiation failed: client connection to 127.8.18.253:41297: connect: Connection refused (error 111)
I20260810 11:36:31.099373 31385 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.18.253" port: 41297 } attempt: 3
I20260810 11:36:31.100687 31388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:36:31.100185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.253:41297 (local address 127.0.0.1:33452)
0810 11:36:31.100421 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0810 11:36:31.100525 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.253:41297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
I20260810 11:36:31.101938 31388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:36:31.101539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.253:41297 (local address 127.0.0.1:33460)
0810 11:36:31.101715 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0810 11:36:31.101787 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.253:41297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260810 11:36:31.102223 31385 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.18.253:41297: Network error: Client connection negotiation failed: client connection to 127.8.18.253:41297: connect: Connection refused (error 111)
I20260810 11:36:31.245558 31385 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.18.253" port: 41297 } attempt: 4
I20260810 11:36:31.247030 31388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:36:31.246389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.253:41297 (local address 127.0.0.1:33464)
0810 11:36:31.246790 (+ 401us) negotiation.cc:107] Waiting for socket to connect
0810 11:36:31.246887 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.253:41297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":223}
I20260810 11:36:31.248188 31388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:36:31.247788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.253:41297 (local address 127.0.0.1:33476)
0810 11:36:31.247950 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0810 11:36:31.248055 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.253:41297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260810 11:36:31.248394 31385 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.18.253:41297: Network error: Client connection negotiation failed: client connection to 127.8.18.253:41297: connect: Connection refused (error 111)
W20260810 11:36:31.293762 31392 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260810 11:36:31.294178 31392 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:36:31.294315 31392 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:36:31.294457 31392 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:36:31.302331 31392 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 11:36:31.302480 31392 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:36:31.302615 31392 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:36:31.302708 31392 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 11:36:31.302827 31392 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 11:36:31.313505 31392 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.18.212:40305
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786361401922362-8267-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786361401922362-8267-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.8.18.254:34903,127.8.18.253:41297,127.8.18.252:40443
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.253.keytab
--principal=kudu/127.8.18.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.18.253:41297
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786361401922362-8267-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.8.18.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786361401922362-8267-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:36:31.314996 31392 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:36:31.316392 31392 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:36:31.328995 31398 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:36:31.328940 31401 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:36:31.328910 31399 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:36:31.329914 31392 server_base.cc:1034] running on GCE node
I20260810 11:36:31.330832 31392 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:36:31.332358 31392 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:36:31.333602 31392 hybrid_clock.cc:648] HybridClock initialized: now 1786361791333519 us; error 69 us; skew 500 ppm
I20260810 11:36:31.333948 31392 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:36:31 dist-test-slave-srgv krb5kdc[31302](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361791, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:36:31.337314 31392 init.cc:377] Logged in from keytab as kudu/127.8.18.253@KRBTEST.COM (short username kudu)
I20260810 11:36:31.338861 31392 webserver.cc:460] Webserver started at http://127.8.18.253:35213/ using document root <none> and password file <none>
I20260810 11:36:31.339603 31392 fs_manager.cc:359] Metadata directory not provided
I20260810 11:36:31.339746 31392 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:36:31.340094 31392 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:36:31.343322 31392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786361401922362-8267-0/minicluster-data/master-1/data/instance:
uuid: "b0fd488a5cc74b05994bca74bf8437c8"
format_stamp: "Formatted at 2026-08-10 11:36:31 on dist-test-slave-srgv"
server_key: "651c53502ea97ab0b17eab05b05d5fc0"
server_key_iv: "bcb54202a3f486441f30c2df50f4edc9"
server_key_version: "encryptionkey@0"
I20260810 11:36:31.344193 31392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786361401922362-8267-0/minicluster-data/master-1/wal/instance:
uuid: "b0fd488a5cc74b05994bca74bf8437c8"
format_stamp: "Formatted at 2026-08-10 11:36:31 on dist-test-slave-srgv"
server_key: "651c53502ea97ab0b17eab05b05d5fc0"
server_key_iv: "bcb54202a3f486441f30c2df50f4edc9"
server_key_version: "encryptionkey@0"
I20260810 11:36:31.349934 31392 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260810 11:36:31.353632 31408 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:36:31.355082 31392 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 11:36:31.355305 31392 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786361401922362-8267-0/minicluster-data/master-1/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786361401922362-8267-0/minicluster-data/master-1/wal
uuid: "b0fd488a5cc74b05994bca74bf8437c8"
format_stamp: "Formatted at 2026-08-10 11:36:31 on dist-test-slave-srgv"
server_key: "651c53502ea97ab0b17eab05b05d5fc0"
server_key_iv: "bcb54202a3f486441f30c2df50f4edc9"
server_key_version: "encryptionkey@0"
I20260810 11:36:31.355556 31392 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786361401922362-8267-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786361401922362-8267-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786361401922362-8267-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:36:31.373610 31392 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:36:31.378290 31392 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:36:31.378619 31392 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:36:31.400441 31392 rpc_server.cc:307] RPC server started. Bound to: 127.8.18.253:41297
I20260810 11:36:31.400460 31460 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.18.253:41297 every 8 connection(s)
I20260810 11:36:31.401993 31392 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786361401922362-8267-0/minicluster-data/master-1/data/info.pb
I20260810 11:36:31.406199 31461 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:36:31.407572 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 31392
I20260810 11:36:31.407797 8267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786361401922362-8267-0/minicluster-data/master-1/wal/instance
I20260810 11:36:31.408281 8267 external_mini_cluster.cc:1428] Setting key 4f36797a0483509a9b54812f9a7775ea
I20260810 11:36:31.413335 31461 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.18.254" port: 34903 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 11:36:31 dist-test-slave-srgv krb5kdc[31302](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361791, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.253@KRBTEST.COM for kudu/127.8.18.254@KRBTEST.COM
I20260810 11:36:31.439569 31463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:36:31.416210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35020 (local address 127.8.18.254:34903)
0810 11:36:31.416773 (+ 563us) server_negotiation.cc:206] Beginning negotiation
0810 11:36:31.416780 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:36:31.417929 (+ 1149us) server_negotiation.cc:407] Connection header received
0810 11:36:31.421005 (+ 3076us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:36:31.421014 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:36:31.421181 (+ 167us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:36:31.421324 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:36:31.423126 (+ 1802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:36:31.424397 (+ 1271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:36:31.426169 (+ 1772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:36:31.426951 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:36:31.427111 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:36:31.431325 (+ 4214us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:36:31.431335 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:36:31.431343 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:36:31.431406 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:36:31.435518 (+ 4112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:36:31.436540 (+ 1022us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:36:31.436549 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:36:31.436555 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:36:31.436738 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:36:31.437326 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:36:31.437334 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:36:31.437338 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:36:31.437798 (+ 460us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:36:31.437988 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0810 11:36:31.438807 (+ 819us) server_negotiation.cc:299] Negotiation successful
0810 11:36:31.439122 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":417,"thread_start_us":281,"threads_started":1}
I20260810 11:36:31.440336 31464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:36:31.416744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:34903 (local address 127.0.0.1:35020)
0810 11:36:31.417692 (+ 948us) negotiation.cc:107] Waiting for socket to connect
0810 11:36:31.417761 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0810 11:36:31.420669 (+ 2908us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:36:31.421661 (+ 992us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:36:31.421671 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:36:31.422265 (+ 594us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:36:31.422868 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:36:31.422886 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:36:31.424595 (+ 1709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:36:31.424600 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:36:31.425993 (+ 1393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:36:31.426015 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:36:31.427213 (+ 1198us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:36:31.427218 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:36:31.427443 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:36:31.428229 (+ 786us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:36:31.428254 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:36:31.431013 (+ 2759us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:36:31.435719 (+ 4706us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:36:31.435727 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:36:31.435743 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:36:31.436365 (+ 622us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:36:31.436883 (+ 518us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:36:31.436887 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:36:31.436890 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:36:31.437162 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:36:31.437964 (+ 802us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:36:31.437971 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:36:31.438370 (+ 399us) client_negotiation.cc:769] Sending connection context
0810 11:36:31.438624 (+ 254us) client_negotiation.cc:240] Negotiation successful
0810 11:36:31.439057 (+ 433us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":600,"thread_start_us":347,"threads_started":1}
WARNING: no policy specified for kudu/127.8.18.252@KRBTEST.COM; defaulting to no policy
I20260810 11:36:31.442509 31461 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.18.253" port: 41297 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.18.252@KRBTEST.COM" created.
Aug 10 11:36:31 dist-test-slave-srgv krb5kdc[31302](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361791, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.253@KRBTEST.COM for kudu/127.8.18.253@KRBTEST.COM
I20260810 11:36:31.461916 31464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:36:31.443368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.253:41297 (local address 127.0.0.1:33486)
0810 11:36:31.443539 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0810 11:36:31.443559 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 11:36:31.443683 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:36:31.444655 (+ 972us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:36:31.444659 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:36:31.445193 (+ 534us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:36:31.445416 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:36:31.445449 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:36:31.447717 (+ 2268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:36:31.447725 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:36:31.449292 (+ 1567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:36:31.449312 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:36:31.450351 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:36:31.450363 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:36:31.450580 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:36:31.451320 (+ 740us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:36:31.451349 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:36:31.454444 (+ 3095us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:36:31.458564 (+ 4120us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:36:31.458568 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:36:31.458571 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:36:31.458972 (+ 401us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:36:31.459507 (+ 535us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:36:31.459511 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:36:31.459514 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:36:31.459707 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:36:31.460593 (+ 886us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:36:31.460599 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:36:31.460807 (+ 208us) client_negotiation.cc:769] Sending connection context
0810 11:36:31.460982 (+ 175us) client_negotiation.cc:240] Negotiation successful
0810 11:36:31.461400 (+ 418us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260810 11:36:31.462127 31467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:36:31.443513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33486 (local address 127.8.18.253:41297)
0810 11:36:31.444181 (+ 668us) server_negotiation.cc:206] Beginning negotiation
0810 11:36:31.444219 (+ 38us) server_negotiation.cc:399] Waiting for connection header
0810 11:36:31.444247 (+ 28us) server_negotiation.cc:407] Connection header received
0810 11:36:31.444298 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:36:31.444304 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:36:31.444489 (+ 185us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:36:31.444649 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:36:31.445670 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:36:31.447551 (+ 1881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:36:31.449527 (+ 1976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:36:31.450156 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:36:31.450263 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:36:31.454785 (+ 4522us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:36:31.454801 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:36:31.454817 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:36:31.454850 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:36:31.458367 (+ 3517us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:36:31.459134 (+ 767us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:36:31.459149 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:36:31.459154 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:36:31.459336 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:36:31.459909 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:36:31.459922 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:36:31.459925 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:36:31.460450 (+ 525us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:36:31.460731 (+ 281us) server_negotiation.cc:1035] Waiting for connection context
0810 11:36:31.461146 (+ 415us) server_negotiation.cc:299] Negotiation successful
0810 11:36:31.461645 (+ 499us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":96,"server-negotiator.queue_time_us":504,"thread_start_us":340,"threads_started":1}
I20260810 11:36:31.464941 31461 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.18.252" port: 40443 } has no permanent_uuid. Determining permanent_uuid...
I20260810 11:36:31.466322 31464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:36:31.465780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.252:40443 (local address 127.0.0.1:44354)
0810 11:36:31.466011 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0810 11:36:31.466156 (+ 145us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.252:40443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
W20260810 11:36:31.466713 31411 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.18.252:40443: connect: Connection refused (error 111)
I20260810 11:36:31.468423 31464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:36:31.468051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.252:40443 (local address 127.0.0.1:44356)
0810 11:36:31.468202 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0810 11:36:31.468290 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.252:40443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260810 11:36:31.468709 31461 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.18.252:40443: Network error: Client connection negotiation failed: client connection to 127.8.18.252:40443: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.18.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.252.keytab.
Entry for principal kudu/127.8.18.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.252.keytab.
I20260810 11:36:31.484952 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786361401922362-8267-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786361401922362-8267-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786361401922362-8267-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786361401922362-8267-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.18.252:40443
--webserver_interface=127.8.18.252
--webserver_port=0
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.252.keytab
--principal=kudu/127.8.18.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.18.212:40305
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.18.254:34903,127.8.18.253:41297,127.8.18.252:40443
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786361401922362-8267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786361401922362-8267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786361401922362-8267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 11:36:31.518079 31461 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.18.252" port: 40443 } attempt: 1
I20260810 11:36:31.519399 31464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:36:31.518860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.252:40443 (local address 127.0.0.1:44370)
0810 11:36:31.519125 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0810 11:36:31.519239 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.252:40443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
I20260810 11:36:31.520447 31464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:36:31.520086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.252:40443 (local address 127.0.0.1:44372)
0810 11:36:31.520261 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0810 11:36:31.520336 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.252:40443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260810 11:36:31.520663 31461 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.18.252:40443: Network error: Client connection negotiation failed: client connection to 127.8.18.252:40443: connect: Connection refused (error 111)
I20260810 11:36:31.547714 31385 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.18.253" port: 41297 } attempt: 5
Aug 10 11:36:31 dist-test-slave-srgv krb5kdc[31302](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361790, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.254@KRBTEST.COM for kudu/127.8.18.253@KRBTEST.COM
I20260810 11:36:31.563921 31388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:36:31.548586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.253:41297 (local address 127.0.0.1:33492)
0810 11:36:31.548836 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0810 11:36:31.548859 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0810 11:36:31.549010 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:36:31.549535 (+ 525us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:36:31.549542 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:36:31.549983 (+ 441us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:36:31.550232 (+ 249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:36:31.550242 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:36:31.551690 (+ 1448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:36:31.551694 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:36:31.552627 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:36:31.552639 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:36:31.553722 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:36:31.553727 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:36:31.553915 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:36:31.554689 (+ 774us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:36:31.554707 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:36:31.557199 (+ 2492us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:36:31.561335 (+ 4136us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:36:31.561340 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:36:31.561342 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:36:31.561849 (+ 507us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:36:31.562294 (+ 445us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:36:31.562297 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:36:31.562299 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:36:31.562437 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:36:31.563094 (+ 657us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:36:31.563098 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:36:31.563259 (+ 161us) client_negotiation.cc:769] Sending connection context
0810 11:36:31.563421 (+ 162us) client_negotiation.cc:240] Negotiation successful
0810 11:36:31.563665 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":88}
I20260810 11:36:31.564345 31467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:36:31.548869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33492 (local address 127.8.18.253:41297)
0810 11:36:31.549120 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0810 11:36:31.549127 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:36:31.549150 (+ 23us) server_negotiation.cc:407] Connection header received
0810 11:36:31.549216 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:36:31.549221 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:36:31.549302 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:36:31.549452 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:36:31.550427 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:36:31.551524 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:36:31.552808 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:36:31.553533 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:36:31.553667 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:36:31.557464 (+ 3797us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:36:31.557472 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:36:31.557477 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:36:31.557522 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:36:31.561165 (+ 3643us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:36:31.561984 (+ 819us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:36:31.561991 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:36:31.561994 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:36:31.562173 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:36:31.562548 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:36:31.562553 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:36:31.562556 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:36:31.562957 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:36:31.563121 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0810 11:36:31.563609 (+ 488us) server_negotiation.cc:299] Negotiation successful
0810 11:36:31.563861 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260810 11:36:31.565514 31385 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.18.252" port: 40443 } has no permanent_uuid. Determining permanent_uuid...
I20260810 11:36:31.566555 31388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:36:31.566177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.252:40443 (local address 127.0.0.1:44376)
0810 11:36:31.566342 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0810 11:36:31.566419 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.252:40443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260810 11:36:31.567517 31388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:36:31.567217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.252:40443 (local address 127.0.0.1:44392)
0810 11:36:31.567349 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0810 11:36:31.567414 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.252:40443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260810 11:36:31.567782 31385 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.18.252:40443: Network error: Client connection negotiation failed: client connection to 127.8.18.252:40443: connect: Connection refused (error 111)
I20260810 11:36:31.588907 31461 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.18.252" port: 40443 } attempt: 2
I20260810 11:36:31.590210 31464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:36:31.589727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.252:40443 (local address 127.0.0.1:44400)
0810 11:36:31.589934 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0810 11:36:31.590038 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.252:40443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260810 11:36:31.591421 31464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:36:31.590982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.252:40443 (local address 127.0.0.1:44410)
0810 11:36:31.591232 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0810 11:36:31.591305 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.252:40443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260810 11:36:31.591691 31461 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.18.252:40443: Network error: Client connection negotiation failed: client connection to 127.8.18.252:40443: connect: Connection refused (error 111)
I20260810 11:36:31.619084 31385 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.18.252" port: 40443 } attempt: 1
I20260810 11:36:31.620383 31388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:36:31.619841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.252:40443 (local address 127.0.0.1:44420)
0810 11:36:31.620084 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0810 11:36:31.620177 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.252:40443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260810 11:36:31.621556 31388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:36:31.621118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.252:40443 (local address 127.0.0.1:44436)
0810 11:36:31.621280 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0810 11:36:31.621362 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.252:40443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260810 11:36:31.621795 31385 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.18.252:40443: Network error: Client connection negotiation failed: client connection to 127.8.18.252:40443: connect: Connection refused (error 111)
I20260810 11:36:31.682934 31461 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.18.252" port: 40443 } attempt: 3
I20260810 11:36:31.684293 31464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:36:31.683786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.252:40443 (local address 127.0.0.1:44448)
0810 11:36:31.684028 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0810 11:36:31.684129 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.252:40443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260810 11:36:31.685595 31464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:36:31.685079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.252:40443 (local address 127.0.0.1:44460)
0810 11:36:31.685317 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0810 11:36:31.685396 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.252:40443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":157}
W20260810 11:36:31.685838 31461 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.18.252:40443: Network error: Client connection negotiation failed: client connection to 127.8.18.252:40443: connect: Connection refused (error 111)
I20260810 11:36:31.690032 31385 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.18.252" port: 40443 } attempt: 2
I20260810 11:36:31.691316 31388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:36:31.690767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.252:40443 (local address 127.0.0.1:44468)
0810 11:36:31.690979 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0810 11:36:31.691083 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.252:40443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260810 11:36:31.692477 31388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:36:31.692068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.252:40443 (local address 127.0.0.1:44474)
0810 11:36:31.692233 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0810 11:36:31.692322 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.252:40443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260810 11:36:31.692737 31385 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.18.252:40443: Network error: Client connection negotiation failed: client connection to 127.8.18.252:40443: connect: Connection refused (error 111)
I20260810 11:36:31.799031 31385 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.18.252" port: 40443 } attempt: 3
I20260810 11:36:31.800370 31388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:36:31.799856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.252:40443 (local address 127.0.0.1:44490)
0810 11:36:31.800086 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0810 11:36:31.800187 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.252:40443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260810 11:36:31.801828 31388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:36:31.801197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.252:40443 (local address 127.0.0.1:44504)
0810 11:36:31.801508 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0810 11:36:31.801645 (+ 137us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.252:40443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116}
W20260810 11:36:31.802110 31385 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.18.252:40443: Network error: Client connection negotiation failed: client connection to 127.8.18.252:40443: connect: Connection refused (error 111)
I20260810 11:36:31.829151 31461 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.18.252" port: 40443 } attempt: 4
I20260810 11:36:31.830514 31464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:36:31.829977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.252:40443 (local address 127.0.0.1:44516)
0810 11:36:31.830252 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0810 11:36:31.830351 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.252:40443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103}
I20260810 11:36:31.831599 31464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:36:31.831236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.252:40443 (local address 127.0.0.1:44524)
0810 11:36:31.831401 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0810 11:36:31.831484 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.252:40443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260810 11:36:31.831811 31461 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.18.252:40443: Network error: Client connection negotiation failed: client connection to 127.8.18.252:40443: connect: Connection refused (error 111)
W20260810 11:36:31.862073 31470 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260810 11:36:31.862511 31470 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:36:31.862648 31470 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:36:31.862788 31470 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:36:31.870370 31470 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 11:36:31.870529 31470 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:36:31.870637 31470 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:36:31.870733 31470 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 11:36:31.870834 31470 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 11:36:31.881466 31470 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.18.212:40305
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786361401922362-8267-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786361401922362-8267-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.8.18.254:34903,127.8.18.253:41297,127.8.18.252:40443
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.252.keytab
--principal=kudu/127.8.18.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.18.252:40443
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786361401922362-8267-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.8.18.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786361401922362-8267-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:36:31.882998 31470 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:36:31.884472 31470 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:36:31.897984 31477 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:36:31.898099 31476 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:36:31.898120 31479 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:36:31.899257 31470 server_base.cc:1034] running on GCE node
I20260810 11:36:31.900173 31470 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:36:31.901925 31470 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:36:31.903209 31470 hybrid_clock.cc:648] HybridClock initialized: now 1786361791903140 us; error 32 us; skew 500 ppm
I20260810 11:36:31.903632 31470 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:36:31 dist-test-slave-srgv krb5kdc[31302](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361791, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:36:31.907392 31470 init.cc:377] Logged in from keytab as kudu/127.8.18.252@KRBTEST.COM (short username kudu)
I20260810 11:36:31.909149 31470 webserver.cc:460] Webserver started at http://127.8.18.252:40483/ using document root <none> and password file <none>
I20260810 11:36:31.909970 31470 fs_manager.cc:359] Metadata directory not provided
I20260810 11:36:31.910081 31470 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:36:31.910429 31470 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:36:31.913692 31470 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786361401922362-8267-0/minicluster-data/master-2/data/instance:
uuid: "39781736efe248a1a157acfe93946339"
format_stamp: "Formatted at 2026-08-10 11:36:31 on dist-test-slave-srgv"
server_key: "c166233cd036a835abf7675af510d137"
server_key_iv: "468847791cf6a181fda7339510063222"
server_key_version: "encryptionkey@0"
I20260810 11:36:31.914522 31470 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786361401922362-8267-0/minicluster-data/master-2/wal/instance:
uuid: "39781736efe248a1a157acfe93946339"
format_stamp: "Formatted at 2026-08-10 11:36:31 on dist-test-slave-srgv"
server_key: "c166233cd036a835abf7675af510d137"
server_key_iv: "468847791cf6a181fda7339510063222"
server_key_version: "encryptionkey@0"
I20260810 11:36:31.920120 31470 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260810 11:36:31.924263 31486 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:36:31.925993 31470 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260810 11:36:31.926220 31470 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786361401922362-8267-0/minicluster-data/master-2/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786361401922362-8267-0/minicluster-data/master-2/wal
uuid: "39781736efe248a1a157acfe93946339"
format_stamp: "Formatted at 2026-08-10 11:36:31 on dist-test-slave-srgv"
server_key: "c166233cd036a835abf7675af510d137"
server_key_iv: "468847791cf6a181fda7339510063222"
server_key_version: "encryptionkey@0"
I20260810 11:36:31.926491 31470 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786361401922362-8267-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786361401922362-8267-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786361401922362-8267-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:36:31.948423 31470 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:36:31.954171 31470 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:36:31.954510 31470 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:36:31.975589 31470 rpc_server.cc:307] RPC server started. Bound to: 127.8.18.252:40443
I20260810 11:36:31.975603 31539 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.18.252:40443 every 8 connection(s)
I20260810 11:36:31.977079 31470 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786361401922362-8267-0/minicluster-data/master-2/data/info.pb
I20260810 11:36:31.978688 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 31470
I20260810 11:36:31.978906 8267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786361401922362-8267-0/minicluster-data/master-2/wal/instance
I20260810 11:36:31.979341 31385 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.18.252" port: 40443 } attempt: 4
I20260810 11:36:31.979529 8267 external_mini_cluster.cc:1428] Setting key eb4c0916fa1c821f81dd4d70df3afb1d
I20260810 11:36:31.982589 31540 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:36:31.994761 31540 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.18.254" port: 34903 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 11:36:31 dist-test-slave-srgv krb5kdc[31302](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361790, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.254@KRBTEST.COM for kudu/127.8.18.252@KRBTEST.COM
Aug 10 11:36:32 dist-test-slave-srgv krb5kdc[31302](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361790, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.18.252@KRBTEST.COM
I20260810 11:36:32.007066 31388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:36:31.980191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.252:40443 (local address 127.0.0.1:44540)
0810 11:36:31.980939 (+ 748us) negotiation.cc:107] Waiting for socket to connect
0810 11:36:31.980962 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0810 11:36:31.981091 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:36:31.987783 (+ 6692us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:36:31.987789 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:36:31.988334 (+ 545us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:36:31.988542 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:36:31.988556 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:36:31.990041 (+ 1485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:36:31.990049 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:36:31.991130 (+ 1081us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:36:31.991149 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:36:31.992664 (+ 1515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:36:31.992672 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:36:31.992978 (+ 306us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:36:31.994910 (+ 1932us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:36:31.994935 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:36:31.998073 (+ 3138us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:36:32.002931 (+ 4858us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:36:32.002937 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:36:32.002940 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:36:32.003569 (+ 629us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:36:32.004211 (+ 642us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:36:32.004216 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:36:32.004219 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:36:32.004427 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:36:32.005672 (+ 1245us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:36:32.005680 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:36:32.005994 (+ 314us) client_negotiation.cc:769] Sending connection context
0810 11:36:32.006196 (+ 202us) client_negotiation.cc:240] Negotiation successful
0810 11:36:32.006562 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":600,"mutex_wait_us":33}
I20260810 11:36:32.008653 31545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:36:31.981359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44540 (local address 127.8.18.252:40443)
0810 11:36:31.987361 (+ 6002us) server_negotiation.cc:206] Beginning negotiation
0810 11:36:31.987369 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:36:31.987401 (+ 32us) server_negotiation.cc:407] Connection header received
0810 11:36:31.987456 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:36:31.987461 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:36:31.987536 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:36:31.987668 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:36:31.988778 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:36:31.989863 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:36:31.991324 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:36:31.992346 (+ 1022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:36:31.993828 (+ 1482us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:36:31.998498 (+ 4670us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:36:31.998512 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:36:31.998533 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:36:31.998598 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:36:32.002740 (+ 4142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:36:32.003728 (+ 988us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:36:32.003737 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:36:32.003747 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:36:32.004032 (+ 285us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:36:32.004582 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:36:32.004587 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:36:32.004590 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:36:32.005497 (+ 907us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:36:32.005698 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0810 11:36:32.006501 (+ 803us) server_negotiation.cc:299] Negotiation successful
0810 11:36:32.007044 (+ 543us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":214,"server-negotiator.queue_time_us":5826,"thread_start_us":5293,"threads_started":1}
I20260810 11:36:32.011155 31544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:36:31.981335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44548 (local address 127.8.18.252:40443)
0810 11:36:31.982355 (+ 1020us) server_negotiation.cc:206] Beginning negotiation
0810 11:36:31.982368 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 11:36:31.982409 (+ 41us) server_negotiation.cc:407] Connection header received
0810 11:36:31.984237 (+ 1828us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:36:31.984261 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:36:31.984662 (+ 401us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:36:31.985086 (+ 424us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:36:31.986719 (+ 1633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:36:31.988752 (+ 2033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:36:31.990738 (+ 1986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:36:31.992250 (+ 1512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:36:31.994617 (+ 2367us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:36:32.003242 (+ 8625us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:36:32.003247 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:36:32.003251 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:36:32.003317 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:36:32.007805 (+ 4488us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:36:32.008634 (+ 829us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:36:32.008640 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:36:32.008643 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:36:32.008848 (+ 205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:36:32.009324 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:36:32.009329 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:36:32.009333 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:36:32.009766 (+ 433us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:36:32.010064 (+ 298us) server_negotiation.cc:1035] Waiting for connection context
0810 11:36:32.010438 (+ 374us) server_negotiation.cc:299] Negotiation successful
0810 11:36:32.010675 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":81,"server-negotiator.queue_time_us":792,"thread_start_us":490,"threads_started":1}
Aug 10 11:36:32 dist-test-slave-srgv krb5kdc[31302](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361791, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.252@KRBTEST.COM for kudu/127.8.18.254@KRBTEST.COM
I20260810 11:36:32.018090 31385 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 06ec21c7adf84c24a54cbaa5cd53c462: Bootstrap starting.
I20260810 11:36:32.022991 31385 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 06ec21c7adf84c24a54cbaa5cd53c462: Neither blocks nor log segments found. Creating new log.
I20260810 11:36:32.023484 31547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:36:31.998838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:34903 (local address 127.0.0.1:35024)
0810 11:36:32.003156 (+ 4318us) negotiation.cc:107] Waiting for socket to connect
0810 11:36:32.003207 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0810 11:36:32.003431 (+ 224us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:36:32.003836 (+ 405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:36:32.003846 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:36:32.005023 (+ 1177us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:36:32.005512 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:36:32.005524 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:36:32.007078 (+ 1554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:36:32.007083 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:36:32.008876 (+ 1793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:36:32.008895 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:36:32.010163 (+ 1268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:36:32.010170 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:36:32.010470 (+ 300us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:36:32.011245 (+ 775us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:36:32.011270 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:36:32.014480 (+ 3210us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:36:32.019619 (+ 5139us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:36:32.019640 (+ 21us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:36:32.019648 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:36:32.020399 (+ 751us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:36:32.020978 (+ 579us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:36:32.020983 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:36:32.020987 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:36:32.021182 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:36:32.022098 (+ 916us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:36:32.022109 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:36:32.022361 (+ 252us) client_negotiation.cc:769] Sending connection context
0810 11:36:32.022593 (+ 232us) client_negotiation.cc:240] Negotiation successful
0810 11:36:32.023003 (+ 410us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1864,"thread_start_us":770,"threads_started":1}
I20260810 11:36:32.023984 31546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:36:31.998824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35024 (local address 127.8.18.254:34903)
0810 11:36:32.000921 (+ 2097us) server_negotiation.cc:206] Beginning negotiation
0810 11:36:32.000928 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:36:32.003348 (+ 2420us) server_negotiation.cc:407] Connection header received
0810 11:36:32.003588 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:36:32.003594 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:36:32.003675 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:36:32.003788 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:36:32.005679 (+ 1891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:36:32.006839 (+ 1160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:36:32.009089 (+ 2250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:36:32.009764 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:36:32.009904 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:36:32.014764 (+ 4860us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:36:32.014771 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:36:32.014776 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:36:32.014829 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:36:32.019369 (+ 4540us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:36:32.020627 (+ 1258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:36:32.020634 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:36:32.020640 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:36:32.020817 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:36:32.021358 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:36:32.021362 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:36:32.021365 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:36:32.021911 (+ 546us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:36:32.022086 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0810 11:36:32.022839 (+ 753us) server_negotiation.cc:299] Negotiation successful
0810 11:36:32.023207 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":39,"server-negotiator.queue_time_us":1901,"thread_start_us":1966,"threads_started":1}
I20260810 11:36:32.025223 31385 log.cc:824] T 00000000000000000000000000000000 P 06ec21c7adf84c24a54cbaa5cd53c462: Log is configured to *not* fsync() on all Append() calls
I20260810 11:36:32.025599 31540 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.18.253" port: 41297 } has no permanent_uuid. Determining permanent_uuid...
I20260810 11:36:32.028708 31385 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 06ec21c7adf84c24a54cbaa5cd53c462: No bootstrap required, opened a new log
Aug 10 11:36:32 dist-test-slave-srgv krb5kdc[31302](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361790, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.18.254@KRBTEST.COM
Aug 10 11:36:32 dist-test-slave-srgv krb5kdc[31302](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361791, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.252@KRBTEST.COM for kudu/127.8.18.253@KRBTEST.COM
I20260810 11:36:32.040328 31385 raft_consensus.cc:348] T 00000000000000000000000000000000 P 06ec21c7adf84c24a54cbaa5cd53c462 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ec21c7adf84c24a54cbaa5cd53c462" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 34903 } } peers { permanent_uuid: "b0fd488a5cc74b05994bca74bf8437c8" member_type: VOTER last_known_addr { host: "127.8.18.253" port: 41297 } } peers { permanent_uuid: "39781736efe248a1a157acfe93946339" member_type: VOTER last_known_addr { host: "127.8.18.252" port: 40443 } }
I20260810 11:36:32.040805 31385 raft_consensus.cc:729] T 00000000000000000000000000000000 P 06ec21c7adf84c24a54cbaa5cd53c462 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06ec21c7adf84c24a54cbaa5cd53c462, State: Initialized, Role: FOLLOWER
I20260810 11:36:32.041354 31385 consensus_queue.cc:260] T 00000000000000000000000000000000 P 06ec21c7adf84c24a54cbaa5cd53c462 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ec21c7adf84c24a54cbaa5cd53c462" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 34903 } } peers { permanent_uuid: "b0fd488a5cc74b05994bca74bf8437c8" member_type: VOTER last_known_addr { host: "127.8.18.253" port: 41297 } } peers { permanent_uuid: "39781736efe248a1a157acfe93946339" member_type: VOTER last_known_addr { host: "127.8.18.252" port: 40443 } }
I20260810 11:36:32.042006 31548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:36:32.014260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35036 (local address 127.8.18.254:34903)
0810 11:36:32.022090 (+ 7830us) server_negotiation.cc:206] Beginning negotiation
0810 11:36:32.022096 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:36:32.022122 (+ 26us) server_negotiation.cc:407] Connection header received
0810 11:36:32.022174 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:36:32.022180 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:36:32.022259 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:36:32.022389 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:36:32.024338 (+ 1949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:36:32.025564 (+ 1226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:36:32.027471 (+ 1907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:36:32.027988 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:36:32.028153 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:36:32.033978 (+ 5825us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:36:32.033987 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:36:32.033997 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:36:32.034041 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:36:32.038202 (+ 4161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:36:32.039185 (+ 983us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:36:32.039196 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:36:32.039200 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:36:32.039405 (+ 205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:36:32.039915 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:36:32.039920 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:36:32.039923 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:36:32.040354 (+ 431us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:36:32.040646 (+ 292us) server_negotiation.cc:1035] Waiting for connection context
0810 11:36:32.041269 (+ 623us) server_negotiation.cc:299] Negotiation successful
0810 11:36:32.041565 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7655,"thread_start_us":7443,"threads_started":1}
I20260810 11:36:32.043262 31467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:36:32.026499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33504 (local address 127.8.18.253:41297)
0810 11:36:32.026688 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0810 11:36:32.026692 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 11:36:32.027065 (+ 373us) server_negotiation.cc:407] Connection header received
0810 11:36:32.027237 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:36:32.027242 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:36:32.027326 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:36:32.027457 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:36:32.029304 (+ 1847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:36:32.030140 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:36:32.031663 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:36:32.032233 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:36:32.032384 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:36:32.036169 (+ 3785us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:36:32.036178 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:36:32.036184 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:36:32.036234 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:36:32.039463 (+ 3229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:36:32.040334 (+ 871us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:36:32.040340 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:36:32.040344 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:36:32.040559 (+ 215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:36:32.041081 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:36:32.041085 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:36:32.041088 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:36:32.041529 (+ 441us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:36:32.041686 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0810 11:36:32.042380 (+ 694us) server_negotiation.cc:299] Negotiation successful
0810 11:36:32.042706 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260810 11:36:32.044538 31547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:36:32.026499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.253:41297 (local address 127.0.0.1:33504)
0810 11:36:32.026952 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0810 11:36:32.026973 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 11:36:32.027096 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:36:32.028392 (+ 1296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:36:32.028397 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:36:32.028976 (+ 579us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:36:32.029135 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:36:32.029145 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:36:32.030289 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:36:32.030295 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:36:32.031449 (+ 1154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:36:32.031461 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:36:32.032624 (+ 1163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:36:32.032632 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:36:32.032829 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:36:32.033548 (+ 719us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:36:32.033576 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:36:32.035889 (+ 2313us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:36:32.039673 (+ 3784us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:36:32.039679 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:36:32.039682 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:36:32.040197 (+ 515us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:36:32.040717 (+ 520us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:36:32.040722 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:36:32.040725 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:36:32.040934 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:36:32.041763 (+ 829us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:36:32.041768 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:36:32.041978 (+ 210us) client_negotiation.cc:769] Sending connection context
0810 11:36:32.043505 (+ 1527us) client_negotiation.cc:240] Negotiation successful
0810 11:36:32.043967 (+ 462us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":321}
I20260810 11:36:32.045861 31540 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.18.252" port: 40443 } has no permanent_uuid. Determining permanent_uuid...
I20260810 11:36:32.046237 31551 sys_catalog.cc:455] T 00000000000000000000000000000000 P 06ec21c7adf84c24a54cbaa5cd53c462 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ec21c7adf84c24a54cbaa5cd53c462" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 34903 } } peers { permanent_uuid: "b0fd488a5cc74b05994bca74bf8437c8" member_type: VOTER last_known_addr { host: "127.8.18.253" port: 41297 } } peers { permanent_uuid: "39781736efe248a1a157acfe93946339" member_type: VOTER last_known_addr { host: "127.8.18.252" port: 40443 } } }
I20260810 11:36:32.046660 31551 sys_catalog.cc:458] T 00000000000000000000000000000000 P 06ec21c7adf84c24a54cbaa5cd53c462 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 11:36:32.047902 31385 sys_catalog.cc:564] T 00000000000000000000000000000000 P 06ec21c7adf84c24a54cbaa5cd53c462 [sys.catalog]: configured and running, proceeding with master startup.
Aug 10 11:36:32 dist-test-slave-srgv krb5kdc[31302](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361791, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.252@KRBTEST.COM for kudu/127.8.18.252@KRBTEST.COM
Aug 10 11:36:32 dist-test-slave-srgv krb5kdc[31302](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361790, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.18.253@KRBTEST.COM
I20260810 11:36:32.061368 31544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:36:32.047043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44564 (local address 127.8.18.252:40443)
0810 11:36:32.047214 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0810 11:36:32.047223 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 11:36:32.047256 (+ 33us) server_negotiation.cc:407] Connection header received
0810 11:36:32.047394 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:36:32.047399 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:36:32.047467 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:36:32.047576 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:36:32.048384 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:36:32.049103 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:36:32.050740 (+ 1637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:36:32.051176 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:36:32.051318 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:36:32.054797 (+ 3479us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:36:32.054802 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:36:32.054806 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:36:32.054835 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:36:32.058108 (+ 3273us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:36:32.058944 (+ 836us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:36:32.058951 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:36:32.058956 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:36:32.059155 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:36:32.059575 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:36:32.059580 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:36:32.059585 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:36:32.060035 (+ 450us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:36:32.060166 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0810 11:36:32.060700 (+ 534us) server_negotiation.cc:299] Negotiation successful
0810 11:36:32.060991 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260810 11:36:32.062410 31547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:36:32.046610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.252:40443 (local address 127.0.0.1:44564)
0810 11:36:32.047178 (+ 568us) negotiation.cc:107] Waiting for socket to connect
0810 11:36:32.047193 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 11:36:32.047272 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:36:32.047591 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:36:32.047595 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:36:32.048068 (+ 473us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:36:32.048242 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:36:32.048252 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:36:32.049239 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:36:32.049246 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:36:32.050589 (+ 1343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:36:32.050602 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:36:32.051322 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:36:32.051327 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:36:32.051548 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:36:32.052195 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:36:32.052213 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:36:32.054587 (+ 2374us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:36:32.058287 (+ 3700us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:36:32.058293 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:36:32.058298 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:36:32.058825 (+ 527us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:36:32.059294 (+ 469us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:36:32.059298 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:36:32.059302 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:36:32.059451 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:36:32.060252 (+ 801us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:36:32.060256 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:36:32.060433 (+ 177us) client_negotiation.cc:769] Sending connection context
0810 11:36:32.061695 (+ 1262us) client_negotiation.cc:240] Negotiation successful
0810 11:36:32.062052 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":485}
I20260810 11:36:32.067438 31467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:36:32.044468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33508 (local address 127.8.18.253:41297)
0810 11:36:32.044655 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0810 11:36:32.044662 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:36:32.044685 (+ 23us) server_negotiation.cc:407] Connection header received
0810 11:36:32.044740 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:36:32.044745 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:36:32.044811 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:36:32.044909 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:36:32.046948 (+ 2039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:36:32.048309 (+ 1361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:36:32.050527 (+ 2218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:36:32.051467 (+ 940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:36:32.051623 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:36:32.058829 (+ 7206us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:36:32.058837 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:36:32.058841 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:36:32.058890 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:36:32.063028 (+ 4138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:36:32.064096 (+ 1068us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:36:32.064102 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:36:32.064105 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:36:32.064297 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:36:32.064825 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:36:32.064830 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:36:32.064833 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:36:32.065216 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:36:32.065513 (+ 297us) server_negotiation.cc:1035] Waiting for connection context
0810 11:36:32.066779 (+ 1266us) server_negotiation.cc:299] Negotiation successful
0810 11:36:32.067050 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260810 11:36:32.068619 31540 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 39781736efe248a1a157acfe93946339: Bootstrap starting.
I20260810 11:36:32.071982 31540 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 39781736efe248a1a157acfe93946339: Neither blocks nor log segments found. Creating new log.
I20260810 11:36:32.073091 31540 log.cc:824] T 00000000000000000000000000000000 P 39781736efe248a1a157acfe93946339: Log is configured to *not* fsync() on all Append() calls
W20260810 11:36:32.073536 31337 tablet.cc:2369] T 00000000000000000000000000000000 P 06ec21c7adf84c24a54cbaa5cd53c462: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 11:36:32.075080 31562 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 06ec21c7adf84c24a54cbaa5cd53c462: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 11:36:32.075309 31562 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 11:36:32.067709 31315 master_runner.cc:186] Error getting master registration for 127.8.18.253:41297: OK, instance_id { permanent_uuid: "b0fd488a5cc74b05994bca74bf8437c8" instance_seqno: 1786361791394221 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260810 11:36:32.080899 31315 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260810 11:36:32.081347 31540 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 39781736efe248a1a157acfe93946339: No bootstrap required, opened a new log
I20260810 11:36:32.083601 8267 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 11:36:32.087688 31360 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "06ec21c7adf84c24a54cbaa5cd53c462"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35036
I20260810 11:36:32.088287 31360 raft_consensus.cc:482] T 00000000000000000000000000000000 P 06ec21c7adf84c24a54cbaa5cd53c462 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260810 11:36:32.088531 31360 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 06ec21c7adf84c24a54cbaa5cd53c462 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:36:32.091521 31540 raft_consensus.cc:348] T 00000000000000000000000000000000 P 39781736efe248a1a157acfe93946339 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ec21c7adf84c24a54cbaa5cd53c462" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 34903 } } peers { permanent_uuid: "b0fd488a5cc74b05994bca74bf8437c8" member_type: VOTER last_known_addr { host: "127.8.18.253" port: 41297 } } peers { permanent_uuid: "39781736efe248a1a157acfe93946339" member_type: VOTER last_known_addr { host: "127.8.18.252" port: 40443 } }
I20260810 11:36:32.091977 31540 raft_consensus.cc:729] T 00000000000000000000000000000000 P 39781736efe248a1a157acfe93946339 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39781736efe248a1a157acfe93946339, State: Initialized, Role: FOLLOWER
I20260810 11:36:32.092521 31360 raft_consensus.cc:504] T 00000000000000000000000000000000 P 06ec21c7adf84c24a54cbaa5cd53c462 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ec21c7adf84c24a54cbaa5cd53c462" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 34903 } } peers { permanent_uuid: "b0fd488a5cc74b05994bca74bf8437c8" member_type: VOTER last_known_addr { host: "127.8.18.253" port: 41297 } } peers { permanent_uuid: "39781736efe248a1a157acfe93946339" member_type: VOTER last_known_addr { host: "127.8.18.252" port: 40443 } }
I20260810 11:36:32.092578 31540 consensus_queue.cc:260] T 00000000000000000000000000000000 P 39781736efe248a1a157acfe93946339 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ec21c7adf84c24a54cbaa5cd53c462" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 34903 } } peers { permanent_uuid: "b0fd488a5cc74b05994bca74bf8437c8" member_type: VOTER last_known_addr { host: "127.8.18.253" port: 41297 } } peers { permanent_uuid: "39781736efe248a1a157acfe93946339" member_type: VOTER last_known_addr { host: "127.8.18.252" port: 40443 } }
I20260810 11:36:32.093683 31565 sys_catalog.cc:455] T 00000000000000000000000000000000 P 39781736efe248a1a157acfe93946339 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ec21c7adf84c24a54cbaa5cd53c462" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 34903 } } peers { permanent_uuid: "b0fd488a5cc74b05994bca74bf8437c8" member_type: VOTER last_known_addr { host: "127.8.18.253" port: 41297 } } peers { permanent_uuid: "39781736efe248a1a157acfe93946339" member_type: VOTER last_known_addr { host: "127.8.18.252" port: 40443 } } }
I20260810 11:36:32.094206 31565 sys_catalog.cc:458] T 00000000000000000000000000000000 P 39781736efe248a1a157acfe93946339 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 11:36:32.095084 31540 sys_catalog.cc:564] T 00000000000000000000000000000000 P 39781736efe248a1a157acfe93946339 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 11:36:32.095371 31360 leader_election.cc:288] T 00000000000000000000000000000000 P 06ec21c7adf84c24a54cbaa5cd53c462 [CANDIDATE]: Term 1 election: Requested vote from peers b0fd488a5cc74b05994bca74bf8437c8 (127.8.18.253:41297), 39781736efe248a1a157acfe93946339 (127.8.18.252:40443)
I20260810 11:36:32.095707 31514 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "06ec21c7adf84c24a54cbaa5cd53c462" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "39781736efe248a1a157acfe93946339"
I20260810 11:36:32.096196 31514 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 39781736efe248a1a157acfe93946339 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:36:32.100378 31514 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 39781736efe248a1a157acfe93946339 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ec21c7adf84c24a54cbaa5cd53c462 in term 1.
I20260810 11:36:32.101291 31334 leader_election.cc:302] T 00000000000000000000000000000000 P 06ec21c7adf84c24a54cbaa5cd53c462 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06ec21c7adf84c24a54cbaa5cd53c462, 39781736efe248a1a157acfe93946339; no voters:
I20260810 11:36:32.101951 31551 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 06ec21c7adf84c24a54cbaa5cd53c462 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:36:32.112408 31551 raft_consensus.cc:686] T 00000000000000000000000000000000 P 06ec21c7adf84c24a54cbaa5cd53c462 [term 1 LEADER]: Becoming Leader. State: Replica: 06ec21c7adf84c24a54cbaa5cd53c462, State: Running, Role: LEADER
I20260810 11:36:32.113356 31551 consensus_queue.cc:237] T 00000000000000000000000000000000 P 06ec21c7adf84c24a54cbaa5cd53c462 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ec21c7adf84c24a54cbaa5cd53c462" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 34903 } } peers { permanent_uuid: "b0fd488a5cc74b05994bca74bf8437c8" member_type: VOTER last_known_addr { host: "127.8.18.253" port: 41297 } } peers { permanent_uuid: "39781736efe248a1a157acfe93946339" member_type: VOTER last_known_addr { host: "127.8.18.252" port: 40443 } }
I20260810 11:36:32.121984 31569 sys_catalog.cc:455] T 00000000000000000000000000000000 P 06ec21c7adf84c24a54cbaa5cd53c462 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 06ec21c7adf84c24a54cbaa5cd53c462. Latest consensus state: current_term: 1 leader_uuid: "06ec21c7adf84c24a54cbaa5cd53c462" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ec21c7adf84c24a54cbaa5cd53c462" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 34903 } } peers { permanent_uuid: "b0fd488a5cc74b05994bca74bf8437c8" member_type: VOTER last_known_addr { host: "127.8.18.253" port: 41297 } } peers { permanent_uuid: "39781736efe248a1a157acfe93946339" member_type: VOTER last_known_addr { host: "127.8.18.252" port: 40443 } } }
I20260810 11:36:32.122598 31569 sys_catalog.cc:458] T 00000000000000000000000000000000 P 06ec21c7adf84c24a54cbaa5cd53c462 [sys.catalog]: This master's current role is: LEADER
I20260810 11:36:32.129119 31574 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 11:36:32.130995 31461 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.18.252" port: 40443 } attempt: 5
Aug 10 11:36:32 dist-test-slave-srgv krb5kdc[31302](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361791, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.253@KRBTEST.COM for kudu/127.8.18.252@KRBTEST.COM
I20260810 11:36:32.156167 31464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:36:32.131821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.252:40443 (local address 127.0.0.1:44574)
0810 11:36:32.132162 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0810 11:36:32.132189 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0810 11:36:32.132351 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:36:32.132932 (+ 581us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:36:32.132938 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:36:32.133538 (+ 600us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:36:32.133769 (+ 231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:36:32.133784 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:36:32.135351 (+ 1567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:36:32.135357 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:36:32.136931 (+ 1574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:36:32.136951 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:36:32.138414 (+ 1463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:36:32.138422 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:36:32.138676 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:36:32.139427 (+ 751us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:36:32.139455 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:36:32.146325 (+ 6870us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:36:32.151464 (+ 5139us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:36:32.151470 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:36:32.151475 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:36:32.152097 (+ 622us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:36:32.153531 (+ 1434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:36:32.153542 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:36:32.153546 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:36:32.153763 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:36:32.155053 (+ 1290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:36:32.155059 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:36:32.155245 (+ 186us) client_negotiation.cc:769] Sending connection context
0810 11:36:32.155396 (+ 151us) client_negotiation.cc:240] Negotiation successful
0810 11:36:32.155743 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179}
I20260810 11:36:32.158126 31544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:36:32.131842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44574 (local address 127.8.18.252:40443)
0810 11:36:32.132073 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0810 11:36:32.132080 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:36:32.132262 (+ 182us) server_negotiation.cc:407] Connection header received
0810 11:36:32.132491 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:36:32.132497 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:36:32.132590 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:36:32.132715 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:36:32.133975 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:36:32.135161 (+ 1186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:36:32.137160 (+ 1999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:36:32.137946 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:36:32.138131 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:36:32.147083 (+ 8952us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:36:32.147091 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:36:32.147097 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:36:32.147146 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:36:32.151201 (+ 4055us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:36:32.152626 (+ 1425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:36:32.152634 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:36:32.152638 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:36:32.152810 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:36:32.153917 (+ 1107us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:36:32.153922 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:36:32.153926 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:36:32.154365 (+ 439us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:36:32.154574 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0810 11:36:32.157334 (+ 2760us) server_negotiation.cc:299] Negotiation successful
0810 11:36:32.157671 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":66}
I20260810 11:36:32.165529 31574 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 11:36:32.167856 31461 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b0fd488a5cc74b05994bca74bf8437c8: Bootstrap starting.
I20260810 11:36:32.172778 31461 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b0fd488a5cc74b05994bca74bf8437c8: Neither blocks nor log segments found. Creating new log.
I20260810 11:36:32.174981 31461 log.cc:824] T 00000000000000000000000000000000 P b0fd488a5cc74b05994bca74bf8437c8: Log is configured to *not* fsync() on all Append() calls
I20260810 11:36:32.177911 31586 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260810 11:36:32.185297 31470 master_runner.cc:186] Error getting master registration for 127.8.18.253:41297: OK, instance_id { permanent_uuid: "b0fd488a5cc74b05994bca74bf8437c8" instance_seqno: 1786361791394221 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260810 11:36:32.198602 31548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:36:32.137971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35052 (local address 127.8.18.254:34903)
0810 11:36:32.138186 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0810 11:36:32.138193 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:36:32.150764 (+ 12571us) server_negotiation.cc:407] Connection header received
0810 11:36:32.151319 (+ 555us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:36:32.151326 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:36:32.151437 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:36:32.151543 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:36:32.172752 (+ 21209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:36:32.174274 (+ 1522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:36:32.181548 (+ 7274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:36:32.182282 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:36:32.182425 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:36:32.189603 (+ 7178us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:36:32.189613 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:36:32.189620 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:36:32.189674 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:36:32.193735 (+ 4061us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:36:32.195404 (+ 1669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:36:32.195411 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:36:32.195414 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:36:32.195645 (+ 231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:36:32.196373 (+ 728us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:36:32.196378 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:36:32.196381 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:36:32.196854 (+ 473us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:36:32.197081 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0810 11:36:32.197950 (+ 869us) server_negotiation.cc:299] Negotiation successful
0810 11:36:32.198199 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260810 11:36:32.199697 31467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:36:32.142923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33516 (local address 127.8.18.253:41297)
0810 11:36:32.143132 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0810 11:36:32.143138 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:36:32.165459 (+ 22321us) server_negotiation.cc:407] Connection header received
0810 11:36:32.165540 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:36:32.165546 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:36:32.165647 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:36:32.165788 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:36:32.168525 (+ 2737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:36:32.170199 (+ 1674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:36:32.176970 (+ 6771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:36:32.177878 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:36:32.178000 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:36:32.188285 (+ 10285us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:36:32.188295 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:36:32.188300 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:36:32.188351 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:36:32.192579 (+ 4228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:36:32.195385 (+ 2806us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:36:32.195392 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:36:32.195396 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:36:32.195610 (+ 214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:36:32.196223 (+ 613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:36:32.196229 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:36:32.196232 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:36:32.196748 (+ 516us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:36:32.197008 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0810 11:36:32.197787 (+ 779us) server_negotiation.cc:299] Negotiation successful
0810 11:36:32.198901 (+ 1114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260810 11:36:32.199463 31514 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 39781736efe248a1a157acfe93946339 [term 1 FOLLOWER]: Refusing update from remote peer 06ec21c7adf84c24a54cbaa5cd53c462: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 11:36:32.202920 31470 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260810 11:36:32.202862 31551 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 06ec21c7adf84c24a54cbaa5cd53c462 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39781736efe248a1a157acfe93946339" member_type: VOTER last_known_addr { host: "127.8.18.252" port: 40443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260810 11:36:32.206907 31492 tablet.cc:2369] T 00000000000000000000000000000000 P 39781736efe248a1a157acfe93946339: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 11:36:32.211403 31545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:36:32.149141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44576 (local address 127.8.18.252:40443)
0810 11:36:32.149480 (+ 339us) server_negotiation.cc:206] Beginning negotiation
0810 11:36:32.149489 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 11:36:32.162902 (+ 13413us) server_negotiation.cc:407] Connection header received
0810 11:36:32.163410 (+ 508us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:36:32.163419 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:36:32.163534 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:36:32.163628 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:36:32.179836 (+ 16208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:36:32.180980 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:36:32.183322 (+ 2342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:36:32.184143 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:36:32.184297 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:36:32.201816 (+ 17519us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:36:32.201823 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:36:32.201829 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:36:32.201882 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:36:32.206774 (+ 4892us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:36:32.207839 (+ 1065us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:36:32.207844 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:36:32.207848 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:36:32.208029 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:36:32.208591 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:36:32.208599 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:36:32.208603 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:36:32.209066 (+ 463us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:36:32.209374 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0810 11:36:32.210654 (+ 1280us) server_negotiation.cc:299] Negotiation successful
0810 11:36:32.210910 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260810 11:36:32.223816 31461 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b0fd488a5cc74b05994bca74bf8437c8: No bootstrap required, opened a new log
I20260810 11:36:32.226935 31565 sys_catalog.cc:455] T 00000000000000000000000000000000 P 39781736efe248a1a157acfe93946339 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 06ec21c7adf84c24a54cbaa5cd53c462. Latest consensus state: current_term: 1 leader_uuid: "06ec21c7adf84c24a54cbaa5cd53c462" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ec21c7adf84c24a54cbaa5cd53c462" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 34903 } } peers { permanent_uuid: "b0fd488a5cc74b05994bca74bf8437c8" member_type: VOTER last_known_addr { host: "127.8.18.253" port: 41297 } } peers { permanent_uuid: "39781736efe248a1a157acfe93946339" member_type: VOTER last_known_addr { host: "127.8.18.252" port: 40443 } } }
I20260810 11:36:32.227285 31565 sys_catalog.cc:458] T 00000000000000000000000000000000 P 39781736efe248a1a157acfe93946339 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 11:36:32.235582 31461 raft_consensus.cc:348] T 00000000000000000000000000000000 P b0fd488a5cc74b05994bca74bf8437c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ec21c7adf84c24a54cbaa5cd53c462" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 34903 } } peers { permanent_uuid: "b0fd488a5cc74b05994bca74bf8437c8" member_type: VOTER last_known_addr { host: "127.8.18.253" port: 41297 } } peers { permanent_uuid: "39781736efe248a1a157acfe93946339" member_type: VOTER last_known_addr { host: "127.8.18.252" port: 40443 } }
I20260810 11:36:32.236044 31461 raft_consensus.cc:729] T 00000000000000000000000000000000 P b0fd488a5cc74b05994bca74bf8437c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0fd488a5cc74b05994bca74bf8437c8, State: Initialized, Role: FOLLOWER
I20260810 11:36:32.236613 31461 consensus_queue.cc:260] T 00000000000000000000000000000000 P b0fd488a5cc74b05994bca74bf8437c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ec21c7adf84c24a54cbaa5cd53c462" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 34903 } } peers { permanent_uuid: "b0fd488a5cc74b05994bca74bf8437c8" member_type: VOTER last_known_addr { host: "127.8.18.253" port: 41297 } } peers { permanent_uuid: "39781736efe248a1a157acfe93946339" member_type: VOTER last_known_addr { host: "127.8.18.252" port: 40443 } }
I20260810 11:36:32.237313 31589 mvcc.cc:205] Tried to move back new op lower bound from 7316937900835905536 to 7316937900504367104. Current Snapshot: MvccSnapshot[applied={T|T < 7316937900835905536}]
I20260810 11:36:32.238389 31593 sys_catalog.cc:455] T 00000000000000000000000000000000 P b0fd488a5cc74b05994bca74bf8437c8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ec21c7adf84c24a54cbaa5cd53c462" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 34903 } } peers { permanent_uuid: "b0fd488a5cc74b05994bca74bf8437c8" member_type: VOTER last_known_addr { host: "127.8.18.253" port: 41297 } } peers { permanent_uuid: "39781736efe248a1a157acfe93946339" member_type: VOTER last_known_addr { host: "127.8.18.252" port: 40443 } } }
I20260810 11:36:32.239387 31593 sys_catalog.cc:458] T 00000000000000000000000000000000 P b0fd488a5cc74b05994bca74bf8437c8 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 11:36:32.240242 31461 sys_catalog.cc:564] T 00000000000000000000000000000000 P b0fd488a5cc74b05994bca74bf8437c8 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 11:36:32.240710 31434 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b0fd488a5cc74b05994bca74bf8437c8 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:36:32.240267 31435 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "06ec21c7adf84c24a54cbaa5cd53c462" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "b0fd488a5cc74b05994bca74bf8437c8"
I20260810 11:36:32.249886 31434 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b0fd488a5cc74b05994bca74bf8437c8 [term 1 FOLLOWER]: Refusing update from remote peer 06ec21c7adf84c24a54cbaa5cd53c462: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 11:36:32.255328 31435 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b0fd488a5cc74b05994bca74bf8437c8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06ec21c7adf84c24a54cbaa5cd53c462 in term 1.
I20260810 11:36:32.255339 31569 sys_catalog.cc:455] T 00000000000000000000000000000000 P 06ec21c7adf84c24a54cbaa5cd53c462 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "06ec21c7adf84c24a54cbaa5cd53c462" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ec21c7adf84c24a54cbaa5cd53c462" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 34903 } } peers { permanent_uuid: "b0fd488a5cc74b05994bca74bf8437c8" member_type: VOTER last_known_addr { host: "127.8.18.253" port: 41297 } } peers { permanent_uuid: "39781736efe248a1a157acfe93946339" member_type: VOTER last_known_addr { host: "127.8.18.252" port: 40443 } } }
I20260810 11:36:32.255661 31569 sys_catalog.cc:458] T 00000000000000000000000000000000 P 06ec21c7adf84c24a54cbaa5cd53c462 [sys.catalog]: This master's current role is: LEADER
I20260810 11:36:32.259567 31574 catalog_manager.cc:1347] Generated new cluster ID: c86e023e24834cf88932c8279225377c
I20260810 11:36:32.259716 31574 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 11:36:32.260357 31565 sys_catalog.cc:455] T 00000000000000000000000000000000 P 39781736efe248a1a157acfe93946339 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "06ec21c7adf84c24a54cbaa5cd53c462" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ec21c7adf84c24a54cbaa5cd53c462" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 34903 } } peers { permanent_uuid: "b0fd488a5cc74b05994bca74bf8437c8" member_type: VOTER last_known_addr { host: "127.8.18.253" port: 41297 } } peers { permanent_uuid: "39781736efe248a1a157acfe93946339" member_type: VOTER last_known_addr { host: "127.8.18.252" port: 40443 } } }
I20260810 11:36:32.260761 31565 sys_catalog.cc:458] T 00000000000000000000000000000000 P 39781736efe248a1a157acfe93946339 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 11:36:32.261185 31551 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 06ec21c7adf84c24a54cbaa5cd53c462 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b0fd488a5cc74b05994bca74bf8437c8" member_type: VOTER last_known_addr { host: "127.8.18.253" port: 41297 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 11:36:32.294142 31574 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 11:36:32.296236 31574 catalog_manager.cc:1504] Loading token signing keys...
I20260810 11:36:32.314251 31574 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 06ec21c7adf84c24a54cbaa5cd53c462: Generated new TSK 0
I20260810 11:36:32.315502 31574 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 11:36:32.315241 31593 sys_catalog.cc:455] T 00000000000000000000000000000000 P b0fd488a5cc74b05994bca74bf8437c8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 06ec21c7adf84c24a54cbaa5cd53c462. Latest consensus state: current_term: 1 leader_uuid: "06ec21c7adf84c24a54cbaa5cd53c462" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ec21c7adf84c24a54cbaa5cd53c462" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 34903 } } peers { permanent_uuid: "b0fd488a5cc74b05994bca74bf8437c8" member_type: VOTER last_known_addr { host: "127.8.18.253" port: 41297 } } peers { permanent_uuid: "39781736efe248a1a157acfe93946339" member_type: VOTER last_known_addr { host: "127.8.18.252" port: 40443 } } }
I20260810 11:36:32.315732 31593 sys_catalog.cc:458] T 00000000000000000000000000000000 P b0fd488a5cc74b05994bca74bf8437c8 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 11:36:32.316259 31600 sys_catalog.cc:455] T 00000000000000000000000000000000 P b0fd488a5cc74b05994bca74bf8437c8 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "06ec21c7adf84c24a54cbaa5cd53c462" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ec21c7adf84c24a54cbaa5cd53c462" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 34903 } } peers { permanent_uuid: "b0fd488a5cc74b05994bca74bf8437c8" member_type: VOTER last_known_addr { host: "127.8.18.253" port: 41297 } } peers { permanent_uuid: "39781736efe248a1a157acfe93946339" member_type: VOTER last_known_addr { host: "127.8.18.252" port: 40443 } } }
I20260810 11:36:32.316879 31600 sys_catalog.cc:458] T 00000000000000000000000000000000 P b0fd488a5cc74b05994bca74bf8437c8 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 11:36:32.318608 31551 sys_catalog.cc:455] T 00000000000000000000000000000000 P 06ec21c7adf84c24a54cbaa5cd53c462 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "06ec21c7adf84c24a54cbaa5cd53c462" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06ec21c7adf84c24a54cbaa5cd53c462" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 34903 } } peers { permanent_uuid: "b0fd488a5cc74b05994bca74bf8437c8" member_type: VOTER last_known_addr { host: "127.8.18.253" port: 41297 } } peers { permanent_uuid: "39781736efe248a1a157acfe93946339" member_type: VOTER last_known_addr { host: "127.8.18.252" port: 40443 } } }
I20260810 11:36:32.319017 31551 sys_catalog.cc:458] T 00000000000000000000000000000000 P 06ec21c7adf84c24a54cbaa5cd53c462 [sys.catalog]: This master's current role is: LEADER
I20260810 11:36:32.339143 31609 catalog_manager.cc:1259] Loaded cluster ID: c86e023e24834cf88932c8279225377c
I20260810 11:36:32.339278 31609 catalog_manager.cc:1552] T 00000000000000000000000000000000 P b0fd488a5cc74b05994bca74bf8437c8: loading cluster ID for follower catalog manager: success
I20260810 11:36:32.343489 31609 catalog_manager.cc:1574] T 00000000000000000000000000000000 P b0fd488a5cc74b05994bca74bf8437c8: acquiring CA information for follower catalog manager: success
I20260810 11:36:32.346891 31609 catalog_manager.cc:1602] T 00000000000000000000000000000000 P b0fd488a5cc74b05994bca74bf8437c8: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260810 11:36:32.451859 31548 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:36:32.436395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35056 (local address 127.8.18.254:34903)
0810 11:36:32.436636 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0810 11:36:32.436643 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:36:32.449626 (+ 12983us) server_negotiation.cc:407] Connection header received
0810 11:36:32.449979 (+ 353us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:36:32.449987 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:36:32.450108 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:36:32.450191 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:36:32.451516 (+ 1325us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35056: BlockingRecv error: recv got EOF from 127.0.0.1:35056 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
W20260810 11:36:32.452939 31467 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:36:32.436031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33526 (local address 127.8.18.253:41297)
0810 11:36:32.436293 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0810 11:36:32.436302 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 11:36:32.449713 (+ 13411us) server_negotiation.cc:407] Connection header received
0810 11:36:32.451208 (+ 1495us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:36:32.451215 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:36:32.451330 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:36:32.451422 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:36:32.452696 (+ 1274us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33526: BlockingRecv error: recv got EOF from 127.0.0.1:33526 (error 108)
Metrics: {"server-negotiator.queue_time_us":96}
W20260810 11:36:32.454463 31545 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:36:32.450354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44580 (local address 127.8.18.252:40443)
0810 11:36:32.450548 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0810 11:36:32.450554 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:36:32.452542 (+ 1988us) server_negotiation.cc:407] Connection header received
0810 11:36:32.452801 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:36:32.452807 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:36:32.452890 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:36:32.452959 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:36:32.454194 (+ 1235us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44580: BlockingRecv error: recv got EOF from 127.0.0.1:44580 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
I20260810 11:36:32.472725 31467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:36:32.464224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33528 (local address 127.8.18.253:41297)
0810 11:36:32.464429 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0810 11:36:32.464437 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:36:32.465026 (+ 589us) server_negotiation.cc:407] Connection header received
0810 11:36:32.465294 (+ 268us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:36:32.465301 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:36:32.465556 (+ 255us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:36:32.465746 (+ 190us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 11:36:32.466272 (+ 526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:36:32.467444 (+ 1172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:36:32.469605 (+ 2161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:36:32.470346 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:36:32.470504 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:36:32.471189 (+ 685us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 11:36:32.471923 (+ 734us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 11:36:32.472109 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0810 11:36:32.472220 (+ 111us) server_negotiation.cc:299] Negotiation successful
0810 11:36:32.472346 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
W20260810 11:36:32.492586 31545 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:36:32.483789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44588 (local address 127.8.18.252:40443)
0810 11:36:32.484030 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0810 11:36:32.484037 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:36:32.490245 (+ 6208us) server_negotiation.cc:407] Connection header received
0810 11:36:32.490709 (+ 464us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:36:32.490716 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:36:32.490866 (+ 150us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:36:32.490949 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:36:32.492316 (+ 1367us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44588: BlockingRecv error: recv got EOF from 127.0.0.1:44588 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
I20260810 11:36:32.505635 31467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:36:32.497801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33540 (local address 127.8.18.253:41297)
0810 11:36:32.498017 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0810 11:36:32.498024 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:36:32.498679 (+ 655us) server_negotiation.cc:407] Connection header received
0810 11:36:32.498890 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:36:32.498898 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:36:32.499033 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:36:32.499157 (+ 124us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 11:36:32.499670 (+ 513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:36:32.500894 (+ 1224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:36:32.503127 (+ 2233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:36:32.503839 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:36:32.503985 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:36:32.504550 (+ 565us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 11:36:32.504727 (+ 177us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 11:36:32.504828 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0810 11:36:32.505009 (+ 181us) server_negotiation.cc:299] Negotiation successful
0810 11:36:32.505114 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
W20260810 11:36:32.518878 31545 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:36:32.516097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44592 (local address 127.8.18.252:40443)
0810 11:36:32.516290 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0810 11:36:32.516296 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:36:32.516891 (+ 595us) server_negotiation.cc:407] Connection header received
0810 11:36:32.517225 (+ 334us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:36:32.517232 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:36:32.517330 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:36:32.517453 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:36:32.518609 (+ 1156us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44592: BlockingRecv error: recv got EOF from 127.0.0.1:44592 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
I20260810 11:36:32.534096 31467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:36:32.525980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33544 (local address 127.8.18.253:41297)
0810 11:36:32.526177 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0810 11:36:32.526184 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:36:32.526816 (+ 632us) server_negotiation.cc:407] Connection header received
0810 11:36:32.527045 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:36:32.527052 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:36:32.527155 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:36:32.527286 (+ 131us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 11:36:32.527853 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:36:32.529094 (+ 1241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:36:32.531566 (+ 2472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:36:32.532373 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:36:32.532546 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:36:32.533111 (+ 565us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 11:36:32.533286 (+ 175us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 11:36:32.533398 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0810 11:36:32.533645 (+ 247us) server_negotiation.cc:299] Negotiation successful
0810 11:36:32.533745 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
W20260810 11:36:32.544368 31545 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:36:32.541687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44600 (local address 127.8.18.252:40443)
0810 11:36:32.541880 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0810 11:36:32.541887 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:36:32.542482 (+ 595us) server_negotiation.cc:407] Connection header received
0810 11:36:32.542838 (+ 356us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:36:32.542845 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:36:32.542943 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:36:32.543022 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:36:32.544133 (+ 1111us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44600: BlockingRecv error: recv got EOF from 127.0.0.1:44600 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
I20260810 11:36:32.561930 31467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:36:32.554142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33548 (local address 127.8.18.253:41297)
0810 11:36:32.554399 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0810 11:36:32.554406 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:36:32.555002 (+ 596us) server_negotiation.cc:407] Connection header received
0810 11:36:32.555225 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:36:32.555232 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:36:32.555359 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:36:32.555469 (+ 110us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 11:36:32.555981 (+ 512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:36:32.557220 (+ 1239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:36:32.559474 (+ 2254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:36:32.560296 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:36:32.560526 (+ 230us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:36:32.561019 (+ 493us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 11:36:32.561167 (+ 148us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 11:36:32.561273 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0810 11:36:32.561483 (+ 210us) server_negotiation.cc:299] Negotiation successful
0810 11:36:32.561593 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124}
W20260810 11:36:32.601940 31545 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:36:32.599425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44604 (local address 127.8.18.252:40443)
0810 11:36:32.599717 (+ 292us) server_negotiation.cc:206] Beginning negotiation
0810 11:36:32.599724 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:36:32.599753 (+ 29us) server_negotiation.cc:407] Connection header received
0810 11:36:32.599954 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:36:32.599962 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:36:32.600070 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:36:32.600159 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:36:32.601671 (+ 1512us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44604: BlockingRecv error: recv got EOF from 127.0.0.1:44604 (error 108)
Metrics: {"server-negotiator.queue_time_us":171}
I20260810 11:36:32.623591 31467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:36:32.615364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33556 (local address 127.8.18.253:41297)
0810 11:36:32.615602 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0810 11:36:32.615610 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:36:32.616286 (+ 676us) server_negotiation.cc:407] Connection header received
0810 11:36:32.616582 (+ 296us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:36:32.616590 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:36:32.616687 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:36:32.616796 (+ 109us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 11:36:32.617384 (+ 588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:36:32.618615 (+ 1231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:36:32.621048 (+ 2433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:36:32.621866 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:36:32.622020 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:36:32.622617 (+ 597us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 11:36:32.622790 (+ 173us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 11:36:32.622888 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0810 11:36:32.623114 (+ 226us) server_negotiation.cc:299] Negotiation successful
0810 11:36:32.623207 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
W20260810 11:36:32.639941 31545 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:36:32.635394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44620 (local address 127.8.18.252:40443)
0810 11:36:32.635616 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0810 11:36:32.635622 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:36:32.636164 (+ 542us) server_negotiation.cc:407] Connection header received
0810 11:36:32.636360 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:36:32.636367 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:36:32.636473 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:36:32.636591 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:36:32.639567 (+ 2976us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44620: BlockingRecv error: recv got EOF from 127.0.0.1:44620 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
I20260810 11:36:32.666096 31467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:36:32.657974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33568 (local address 127.8.18.253:41297)
0810 11:36:32.658209 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0810 11:36:32.658215 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:36:32.658864 (+ 649us) server_negotiation.cc:407] Connection header received
0810 11:36:32.659098 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:36:32.659105 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:36:32.659208 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:36:32.659328 (+ 120us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 11:36:32.659933 (+ 605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:36:32.661258 (+ 1325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:36:32.663424 (+ 2166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:36:32.664262 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:36:32.664425 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:36:32.665072 (+ 647us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 11:36:32.665280 (+ 208us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 11:36:32.665381 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0810 11:36:32.665647 (+ 266us) server_negotiation.cc:299] Negotiation successful
0810 11:36:32.665767 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
W20260810 11:36:32.677619 31545 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:36:32.675068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44636 (local address 127.8.18.252:40443)
0810 11:36:32.675276 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0810 11:36:32.675283 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:36:32.675908 (+ 625us) server_negotiation.cc:407] Connection header received
0810 11:36:32.676130 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:36:32.676136 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:36:32.676239 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:36:32.676361 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:36:32.677326 (+ 965us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44636: BlockingRecv error: recv got EOF from 127.0.0.1:44636 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
I20260810 11:36:32.720575 31467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:36:32.714055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33574 (local address 127.8.18.253:41297)
0810 11:36:32.714275 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0810 11:36:32.714280 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:36:32.714880 (+ 600us) server_negotiation.cc:407] Connection header received
0810 11:36:32.715107 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:36:32.715111 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:36:32.715187 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:36:32.715278 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 11:36:32.715834 (+ 556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:36:32.716609 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:36:32.718643 (+ 2034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:36:32.719139 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:36:32.719240 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:36:32.719763 (+ 523us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 11:36:32.719880 (+ 117us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 11:36:32.719951 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0810 11:36:32.720134 (+ 183us) server_negotiation.cc:299] Negotiation successful
0810 11:36:32.720205 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
W20260810 11:36:32.729844 31545 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:36:32.727536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44652 (local address 127.8.18.252:40443)
0810 11:36:32.727719 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0810 11:36:32.727723 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 11:36:32.728152 (+ 429us) server_negotiation.cc:407] Connection header received
0810 11:36:32.728369 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:36:32.728373 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:36:32.728447 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:36:32.728534 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:36:32.729614 (+ 1080us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44652: BlockingRecv error: recv got EOF from 127.0.0.1:44652 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
I20260810 11:36:32.804716 31467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:36:32.798087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33580 (local address 127.8.18.253:41297)
0810 11:36:32.798378 (+ 291us) server_negotiation.cc:206] Beginning negotiation
0810 11:36:32.798386 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:36:32.798768 (+ 382us) server_negotiation.cc:407] Connection header received
0810 11:36:32.798969 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:36:32.798973 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:36:32.799046 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:36:32.799130 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 11:36:32.799677 (+ 547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:36:32.800464 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:36:32.802629 (+ 2165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:36:32.803251 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:36:32.803368 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:36:32.803943 (+ 575us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 11:36:32.804086 (+ 143us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 11:36:32.804161 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0810 11:36:32.804351 (+ 190us) server_negotiation.cc:299] Negotiation successful
0810 11:36:32.804425 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
W20260810 11:36:32.814021 31545 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:36:32.811594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44666 (local address 127.8.18.252:40443)
0810 11:36:32.811876 (+ 282us) server_negotiation.cc:206] Beginning negotiation
0810 11:36:32.811886 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 11:36:32.812411 (+ 525us) server_negotiation.cc:407] Connection header received
0810 11:36:32.812606 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:36:32.812610 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:36:32.812697 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:36:32.812797 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:36:32.813749 (+ 952us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44666: BlockingRecv error: recv got EOF from 127.0.0.1:44666 (error 108)
Metrics: {"server-negotiator.queue_time_us":95}
I20260810 11:36:32.952463 31467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:36:32.946482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33594 (local address 127.8.18.253:41297)
0810 11:36:32.946770 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0810 11:36:32.946778 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:36:32.947203 (+ 425us) server_negotiation.cc:407] Connection header received
0810 11:36:32.947344 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:36:32.947348 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:36:32.947427 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:36:32.947538 (+ 111us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 11:36:32.948232 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:36:32.949146 (+ 914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:36:32.950644 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:36:32.951089 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:36:32.951198 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:36:32.951709 (+ 511us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 11:36:32.951836 (+ 127us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 11:36:32.951921 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0810 11:36:32.952084 (+ 163us) server_negotiation.cc:299] Negotiation successful
0810 11:36:32.952160 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100}
W20260810 11:36:32.961814 31545 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:36:32.959461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44678 (local address 127.8.18.252:40443)
0810 11:36:32.959654 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0810 11:36:32.959660 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:36:32.960352 (+ 692us) server_negotiation.cc:407] Connection header received
0810 11:36:32.960461 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:36:32.960465 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:36:32.960544 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:36:32.960640 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:36:32.961572 (+ 932us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44678: BlockingRecv error: recv got EOF from 127.0.0.1:44678 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
I20260810 11:36:33.182780 31586 catalog_manager.cc:1259] Loaded cluster ID: c86e023e24834cf88932c8279225377c
I20260810 11:36:33.182971 31586 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 39781736efe248a1a157acfe93946339: loading cluster ID for follower catalog manager: success
I20260810 11:36:33.186075 31586 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 39781736efe248a1a157acfe93946339: acquiring CA information for follower catalog manager: success
I20260810 11:36:33.188268 31586 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 39781736efe248a1a157acfe93946339: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 11:36:33.235520 31467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:36:33.228622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33602 (local address 127.8.18.253:41297)
0810 11:36:33.228903 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0810 11:36:33.228914 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 11:36:33.229449 (+ 535us) server_negotiation.cc:407] Connection header received
0810 11:36:33.229710 (+ 261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:36:33.229716 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:36:33.229827 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:36:33.229947 (+ 120us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 11:36:33.230419 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:36:33.231627 (+ 1208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:36:33.233257 (+ 1630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:36:33.233970 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:36:33.234121 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:36:33.234633 (+ 512us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 11:36:33.234786 (+ 153us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 11:36:33.234889 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0810 11:36:33.235104 (+ 215us) server_negotiation.cc:299] Negotiation successful
0810 11:36:33.235211 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260810 11:36:33.250654 31545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:36:33.243104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44680 (local address 127.8.18.252:40443)
0810 11:36:33.243366 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0810 11:36:33.243373 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:36:33.243774 (+ 401us) server_negotiation.cc:407] Connection header received
0810 11:36:33.243965 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:36:33.243972 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:36:33.244221 (+ 249us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:36:33.244365 (+ 144us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 11:36:33.244797 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:36:33.246060 (+ 1263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:36:33.247581 (+ 1521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:36:33.248356 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:36:33.248497 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:36:33.249133 (+ 636us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 11:36:33.249942 (+ 809us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 11:36:33.250112 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0810 11:36:33.250176 (+ 64us) server_negotiation.cc:299] Negotiation successful
0810 11:36:33.250285 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260810 11:36:33.280195 31467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:36:33.257800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33618 (local address 127.8.18.253:41297)
0810 11:36:33.258059 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0810 11:36:33.258068 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 11:36:33.270875 (+ 12807us) server_negotiation.cc:407] Connection header received
0810 11:36:33.271503 (+ 628us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:36:33.271512 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:36:33.271662 (+ 150us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:36:33.271752 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 11:36:33.273831 (+ 2079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:36:33.274999 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:36:33.277503 (+ 2504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:36:33.278190 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:36:33.278303 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:36:33.279241 (+ 938us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 11:36:33.279396 (+ 155us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 11:36:33.279468 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0810 11:36:33.279716 (+ 248us) server_negotiation.cc:299] Negotiation successful
0810 11:36:33.279832 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260810 11:36:33.280618 31732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:36:33.257434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35062 (local address 127.8.18.254:34903)
0810 11:36:33.269170 (+ 11736us) server_negotiation.cc:206] Beginning negotiation
0810 11:36:33.269180 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 11:36:33.270139 (+ 959us) server_negotiation.cc:407] Connection header received
0810 11:36:33.272740 (+ 2601us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:36:33.272756 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:36:33.272930 (+ 174us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:36:33.273069 (+ 139us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 11:36:33.274215 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:36:33.275404 (+ 1189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:36:33.277788 (+ 2384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:36:33.278462 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:36:33.278569 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:36:33.279331 (+ 762us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 11:36:33.279841 (+ 510us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 11:36:33.279931 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0810 11:36:33.280158 (+ 227us) server_negotiation.cc:299] Negotiation successful
0810 11:36:33.280283 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11518,"thread_start_us":11774,"threads_started":1}
I20260810 11:36:33.281962 31545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:36:33.270318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44684 (local address 127.8.18.252:40443)
0810 11:36:33.272072 (+ 1754us) server_negotiation.cc:206] Beginning negotiation
0810 11:36:33.272079 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:36:33.273842 (+ 1763us) server_negotiation.cc:407] Connection header received
0810 11:36:33.274182 (+ 340us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:36:33.274191 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:36:33.274292 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:36:33.274364 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 11:36:33.275212 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:36:33.276463 (+ 1251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:36:33.279183 (+ 2720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:36:33.279987 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:36:33.280096 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:36:33.281013 (+ 917us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 11:36:33.281173 (+ 160us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 11:36:33.281241 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0810 11:36:33.281508 (+ 267us) server_negotiation.cc:299] Negotiation successful
0810 11:36:33.281627 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1583}
I20260810 11:36:33.288264 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 31315
I20260810 11:36:33.289754 31380 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:36:33.410476 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 31315
I20260810 11:36:33.443614 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 31392
I20260810 11:36:33.444875 31456 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:36:33.559957 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 31392
I20260810 11:36:33.584478 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 31470
I20260810 11:36:33.585726 31535 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:36:33.701390 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 31470
2026-08-10T11:36:33Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (4851 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786361401922362-8267-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 11:36:33 dist-test-slave-srgv krb5kdc[31741](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 11:36:33 dist-test-slave-srgv krb5kdc[31741](info): set up 2 sockets
Aug 10 11:36:33 dist-test-slave-srgv krb5kdc[31741](info): commencing operation
krb5kdc: starting...
W20260810 11:36:34.868685 8267 mini_kdc.cc:121] Time spent starting KDC: real 1.121s user 0.008s sys 0.036s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786361401922362-8267-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786361401922362-8267-0/krb5kdc/test-user.keytab.
Aug 10 11:36:35 dist-test-slave-srgv krb5kdc[31741](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361795, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T11:36:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T11:36:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.18.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.18.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.18.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.254.keytab.
Entry for principal kudu/127.8.18.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.254.keytab.
I20260810 11:36:35.209195 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786361401922362-8267-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786361401922362-8267-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786361401922362-8267-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.18.254:40371
--webserver_interface=127.8.18.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.254.keytab
--principal=kudu/127.8.18.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.18.212:37737
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.18.254:40371,127.8.18.253:46623,127.8.18.252:45977
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786361401922362-8267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786361401922362-8267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786361401922362-8267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 11:36:35.580260 31754 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260810 11:36:35.580700 31754 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:36:35.580832 31754 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:36:35.580986 31754 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:36:35.589623 31754 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 11:36:35.589745 31754 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:36:35.589848 31754 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:36:35.589948 31754 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 11:36:35.590066 31754 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 11:36:35.600697 31754 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.18.212:37737
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786361401922362-8267-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786361401922362-8267-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.18.254:40371,127.8.18.253:46623,127.8.18.252:45977
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.254.keytab
--principal=kudu/127.8.18.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.18.254:40371
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.18.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786361401922362-8267-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:36:35.602197 31754 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:36:35.603605 31754 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:36:35.616376 31760 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:36:35.616510 31763 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:36:35.616564 31759 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:36:35.617663 31754 server_base.cc:1034] running on GCE node
I20260810 11:36:35.618557 31754 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:36:35.620185 31754 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:36:35.621562 31754 hybrid_clock.cc:648] HybridClock initialized: now 1786361795621472 us; error 44 us; skew 500 ppm
I20260810 11:36:35.621927 31754 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:36:35 dist-test-slave-srgv krb5kdc[31741](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361795, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:36:35.630677 31754 init.cc:377] Logged in from keytab as kudu/127.8.18.254@KRBTEST.COM (short username kudu)
I20260810 11:36:35.632517 31754 webserver.cc:460] Webserver started at http://127.8.18.254:38903/ using document root <none> and password file <none>
I20260810 11:36:35.633394 31754 fs_manager.cc:359] Metadata directory not provided
I20260810 11:36:35.633626 31754 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:36:35.633996 31754 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:36:35.637284 31754 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786361401922362-8267-0/minicluster-data/master-0/data/instance:
uuid: "72340a6f145b46d28372cd7526703a45"
format_stamp: "Formatted at 2026-08-10 11:36:35 on dist-test-slave-srgv"
server_key: "4ba4ddcd1f0cebd3fd298d665012b509"
server_key_iv: "10b5de115a9a0597b6285e94bc860b4d"
server_key_version: "encryptionkey@0"
I20260810 11:36:35.638260 31754 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786361401922362-8267-0/minicluster-data/master-0/wal/instance:
uuid: "72340a6f145b46d28372cd7526703a45"
format_stamp: "Formatted at 2026-08-10 11:36:35 on dist-test-slave-srgv"
server_key: "4ba4ddcd1f0cebd3fd298d665012b509"
server_key_iv: "10b5de115a9a0597b6285e94bc860b4d"
server_key_version: "encryptionkey@0"
I20260810 11:36:35.643940 31754 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260810 11:36:35.647806 31771 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:36:35.649350 31754 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 11:36:35.649607 31754 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786361401922362-8267-0/minicluster-data/master-0/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786361401922362-8267-0/minicluster-data/master-0/wal
uuid: "72340a6f145b46d28372cd7526703a45"
format_stamp: "Formatted at 2026-08-10 11:36:35 on dist-test-slave-srgv"
server_key: "4ba4ddcd1f0cebd3fd298d665012b509"
server_key_iv: "10b5de115a9a0597b6285e94bc860b4d"
server_key_version: "encryptionkey@0"
I20260810 11:36:35.649938 31754 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786361401922362-8267-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786361401922362-8267-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786361401922362-8267-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:36:35.665169 31754 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:36:35.670104 31754 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:36:35.670447 31754 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:36:35.691298 31754 rpc_server.cc:307] RPC server started. Bound to: 127.8.18.254:40371
I20260810 11:36:35.691336 31825 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.18.254:40371 every 8 connection(s)
I20260810 11:36:35.692834 31754 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786361401922362-8267-0/minicluster-data/master-0/data/info.pb
I20260810 11:36:35.693543 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 31754
I20260810 11:36:35.693758 8267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786361401922362-8267-0/minicluster-data/master-0/wal/instance
I20260810 11:36:35.694414 8267 external_mini_cluster.cc:1428] Setting key 618ef7e73526c1f9d703a74c7a389f23
I20260810 11:36:35.697259 31826 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:36:35.704398 31826 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.18.254" port: 40371 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 11:36:35 dist-test-slave-srgv krb5kdc[31741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361795, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.254@KRBTEST.COM for kudu/127.8.18.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.8.18.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.18.253@KRBTEST.COM" created.
I20260810 11:36:35.733541 31830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:36:35.707922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:40371 (local address 127.0.0.1:49166)
0810 11:36:35.709195 (+ 1273us) negotiation.cc:107] Waiting for socket to connect
0810 11:36:35.709263 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0810 11:36:35.712389 (+ 3126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:36:35.713672 (+ 1283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:36:35.713682 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:36:35.714666 (+ 984us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:36:35.715297 (+ 631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:36:35.715329 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:36:35.717178 (+ 1849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:36:35.717187 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:36:35.719495 (+ 2308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:36:35.719523 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:36:35.720823 (+ 1300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:36:35.720836 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:36:35.721140 (+ 304us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:36:35.722001 (+ 861us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:36:35.722048 (+ 47us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:36:35.725299 (+ 3251us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:36:35.728980 (+ 3681us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:36:35.728993 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:36:35.729007 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:36:35.729622 (+ 615us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:36:35.730096 (+ 474us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:36:35.730102 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:36:35.730105 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:36:35.730385 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:36:35.731374 (+ 989us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:36:35.731387 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:36:35.731645 (+ 258us) client_negotiation.cc:769] Sending connection context
0810 11:36:35.731876 (+ 231us) client_negotiation.cc:240] Negotiation successful
0810 11:36:35.732329 (+ 453us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":975,"spinlock_wait_cycles":13184,"thread_start_us":752,"threads_started":1}
I20260810 11:36:35.733568 31829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:36:35.707935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49166 (local address 127.8.18.254:40371)
0810 11:36:35.708864 (+ 929us) server_negotiation.cc:206] Beginning negotiation
0810 11:36:35.708887 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0810 11:36:35.709458 (+ 571us) server_negotiation.cc:407] Connection header received
0810 11:36:35.712879 (+ 3421us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:36:35.712885 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:36:35.713317 (+ 432us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:36:35.713579 (+ 262us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:36:35.715571 (+ 1992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:36:35.716936 (+ 1365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:36:35.719773 (+ 2837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:36:35.720473 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:36:35.720659 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:36:35.725640 (+ 4981us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:36:35.725650 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:36:35.725664 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:36:35.725697 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:36:35.728806 (+ 3109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:36:35.729782 (+ 976us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:36:35.729795 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:36:35.729798 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:36:35.729967 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:36:35.730529 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:36:35.730533 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:36:35.730535 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:36:35.731146 (+ 611us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:36:35.731364 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0810 11:36:35.732098 (+ 734us) server_negotiation.cc:299] Negotiation successful
0810 11:36:35.732560 (+ 462us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":99,"server-negotiator.queue_time_us":642,"thread_start_us":380,"threads_started":1}
I20260810 11:36:35.736855 31826 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.18.253" port: 46623 } has no permanent_uuid. Determining permanent_uuid...
I20260810 11:36:35.738140 31830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:36:35.737677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.253:46623 (local address 127.0.0.1:38012)
0810 11:36:35.737881 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0810 11:36:35.738011 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.253:46623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
W20260810 11:36:35.738445 31772 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.18.253:46623: connect: Connection refused (error 111)
I20260810 11:36:35.740185 31830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:36:35.739797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.253:46623 (local address 127.0.0.1:38018)
0810 11:36:35.739967 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0810 11:36:35.740038 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.253:46623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260810 11:36:35.740423 31826 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.18.253:46623: Network error: Client connection negotiation failed: client connection to 127.8.18.253:46623: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.18.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.253.keytab.
Entry for principal kudu/127.8.18.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.253.keytab.
I20260810 11:36:35.773977 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786361401922362-8267-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786361401922362-8267-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786361401922362-8267-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786361401922362-8267-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.18.253:46623
--webserver_interface=127.8.18.253
--webserver_port=0
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.253.keytab
--principal=kudu/127.8.18.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.18.212:37737
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.18.254:40371,127.8.18.253:46623,127.8.18.252:45977
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786361401922362-8267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786361401922362-8267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786361401922362-8267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 11:36:35.789779 31826 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.18.253" port: 46623 } attempt: 1
I20260810 11:36:35.791175 31830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:36:35.790666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.253:46623 (local address 127.0.0.1:38030)
0810 11:36:35.790890 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0810 11:36:35.791000 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.253:46623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260810 11:36:35.792295 31830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:36:35.791870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.253:46623 (local address 127.0.0.1:38038)
0810 11:36:35.792082 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0810 11:36:35.792164 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.253:46623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260810 11:36:35.792481 31826 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.18.253:46623: Network error: Client connection negotiation failed: client connection to 127.8.18.253:46623: connect: Connection refused (error 111)
I20260810 11:36:35.860745 31826 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.18.253" port: 46623 } attempt: 2
I20260810 11:36:35.862051 31830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:36:35.861603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.253:46623 (local address 127.0.0.1:38048)
0810 11:36:35.861807 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0810 11:36:35.861909 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.253:46623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260810 11:36:35.863251 31830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:36:35.862836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.253:46623 (local address 127.0.0.1:38058)
0810 11:36:35.863000 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0810 11:36:35.863075 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.253:46623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260810 11:36:35.863452 31826 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.18.253:46623: Network error: Client connection negotiation failed: client connection to 127.8.18.253:46623: connect: Connection refused (error 111)
I20260810 11:36:35.954698 31826 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.18.253" port: 46623 } attempt: 3
I20260810 11:36:35.956004 31830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:36:35.955512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.253:46623 (local address 127.0.0.1:38072)
0810 11:36:35.955740 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0810 11:36:35.955850 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.253:46623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
I20260810 11:36:35.957154 31830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:36:35.956756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.253:46623 (local address 127.0.0.1:38076)
0810 11:36:35.956911 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0810 11:36:35.956992 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.253:46623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260810 11:36:35.957329 31826 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.18.253:46623: Network error: Client connection negotiation failed: client connection to 127.8.18.253:46623: connect: Connection refused (error 111)
I20260810 11:36:36.100684 31826 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.18.253" port: 46623 } attempt: 4
I20260810 11:36:36.102144 31830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:36:36.101513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.253:46623 (local address 127.0.0.1:38080)
0810 11:36:36.101891 (+ 378us) negotiation.cc:107] Waiting for socket to connect
0810 11:36:36.101989 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.253:46623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":219}
I20260810 11:36:36.103403 31830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:36:36.102972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.253:46623 (local address 127.0.0.1:38086)
0810 11:36:36.103144 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0810 11:36:36.103257 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.253:46623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260810 11:36:36.103615 31826 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.18.253:46623: Network error: Client connection negotiation failed: client connection to 127.8.18.253:46623: connect: Connection refused (error 111)
W20260810 11:36:36.150785 31833 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260810 11:36:36.151209 31833 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:36:36.151351 31833 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:36:36.151491 31833 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:36:36.161953 31833 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 11:36:36.162107 31833 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:36:36.162242 31833 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:36:36.162338 31833 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 11:36:36.162451 31833 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 11:36:36.173338 31833 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.18.212:37737
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786361401922362-8267-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786361401922362-8267-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.8.18.254:40371,127.8.18.253:46623,127.8.18.252:45977
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.253.keytab
--principal=kudu/127.8.18.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.18.253:46623
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786361401922362-8267-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.8.18.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786361401922362-8267-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:36:36.174846 31833 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:36:36.176298 31833 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:36:36.190758 31839 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:36:36.190871 31838 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:36:36.190976 31841 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:36:36.192144 31833 server_base.cc:1034] running on GCE node
I20260810 11:36:36.193029 31833 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:36:36.194770 31833 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:36:36.196041 31833 hybrid_clock.cc:648] HybridClock initialized: now 1786361796195960 us; error 60 us; skew 500 ppm
I20260810 11:36:36.196460 31833 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:36:36 dist-test-slave-srgv krb5kdc[31741](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361796, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:36:36.200344 31833 init.cc:377] Logged in from keytab as kudu/127.8.18.253@KRBTEST.COM (short username kudu)
I20260810 11:36:36.202124 31833 webserver.cc:460] Webserver started at http://127.8.18.253:38253/ using document root <none> and password file <none>
I20260810 11:36:36.203166 31833 fs_manager.cc:359] Metadata directory not provided
I20260810 11:36:36.203279 31833 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:36:36.203575 31833 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:36:36.206852 31833 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786361401922362-8267-0/minicluster-data/master-1/data/instance:
uuid: "7a1346fc8626434aae9378df2825aecf"
format_stamp: "Formatted at 2026-08-10 11:36:36 on dist-test-slave-srgv"
server_key: "77391dc3ed01995c5970f2a599fc65e5"
server_key_iv: "ba7d30606f8abbb80696e22343e34dc7"
server_key_version: "encryptionkey@0"
I20260810 11:36:36.207677 31833 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786361401922362-8267-0/minicluster-data/master-1/wal/instance:
uuid: "7a1346fc8626434aae9378df2825aecf"
format_stamp: "Formatted at 2026-08-10 11:36:36 on dist-test-slave-srgv"
server_key: "77391dc3ed01995c5970f2a599fc65e5"
server_key_iv: "ba7d30606f8abbb80696e22343e34dc7"
server_key_version: "encryptionkey@0"
I20260810 11:36:36.213554 31833 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260810 11:36:36.217615 31848 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:36:36.219226 31833 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 11:36:36.219445 31833 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786361401922362-8267-0/minicluster-data/master-1/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786361401922362-8267-0/minicluster-data/master-1/wal
uuid: "7a1346fc8626434aae9378df2825aecf"
format_stamp: "Formatted at 2026-08-10 11:36:36 on dist-test-slave-srgv"
server_key: "77391dc3ed01995c5970f2a599fc65e5"
server_key_iv: "ba7d30606f8abbb80696e22343e34dc7"
server_key_version: "encryptionkey@0"
I20260810 11:36:36.219735 31833 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786361401922362-8267-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786361401922362-8267-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786361401922362-8267-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:36:36.264897 31833 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:36:36.269989 31833 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:36:36.270325 31833 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:36:36.291415 31833 rpc_server.cc:307] RPC server started. Bound to: 127.8.18.253:46623
I20260810 11:36:36.291456 31900 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.18.253:46623 every 8 connection(s)
I20260810 11:36:36.292871 31833 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786361401922362-8267-0/minicluster-data/master-1/data/info.pb
I20260810 11:36:36.297051 31901 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:36:36.301834 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 31833
I20260810 11:36:36.302039 8267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786361401922362-8267-0/minicluster-data/master-1/wal/instance
I20260810 11:36:36.302615 8267 external_mini_cluster.cc:1428] Setting key 5d1337e9c72bb376735ad88fb3d64fcf
I20260810 11:36:36.305632 31901 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.18.254" port: 40371 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 11:36:36 dist-test-slave-srgv krb5kdc[31741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361796, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.253@KRBTEST.COM for kudu/127.8.18.254@KRBTEST.COM
I20260810 11:36:36.334002 31903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:36:36.309302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51386 (local address 127.8.18.254:40371)
0810 11:36:36.309953 (+ 651us) server_negotiation.cc:206] Beginning negotiation
0810 11:36:36.309960 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:36:36.311211 (+ 1251us) server_negotiation.cc:407] Connection header received
0810 11:36:36.314723 (+ 3512us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:36:36.314729 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:36:36.314846 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:36:36.314954 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:36:36.317568 (+ 2614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:36:36.318909 (+ 1341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:36:36.320692 (+ 1783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:36:36.321549 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:36:36.321724 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:36:36.325974 (+ 4250us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:36:36.325983 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:36:36.325991 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:36:36.326057 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:36:36.330189 (+ 4132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:36:36.331058 (+ 869us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:36:36.331068 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:36:36.331075 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:36:36.331246 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:36:36.331696 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:36:36.331704 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:36:36.331710 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:36:36.332179 (+ 469us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:36:36.332333 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0810 11:36:36.333135 (+ 802us) server_negotiation.cc:299] Negotiation successful
0810 11:36:36.333461 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":502,"thread_start_us":356,"threads_started":1}
I20260810 11:36:36.334718 31904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:36:36.309988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:40371 (local address 127.0.0.1:51386)
0810 11:36:36.310980 (+ 992us) negotiation.cc:107] Waiting for socket to connect
0810 11:36:36.311043 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0810 11:36:36.314234 (+ 3191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:36:36.315303 (+ 1069us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:36:36.315318 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:36:36.316256 (+ 938us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:36:36.317167 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:36:36.317200 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:36:36.319184 (+ 1984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:36:36.319188 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:36:36.320484 (+ 1296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:36:36.320500 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:36:36.321802 (+ 1302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:36:36.321807 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:36:36.322039 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:36:36.322866 (+ 827us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:36:36.322898 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:36:36.325655 (+ 2757us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:36:36.330377 (+ 4722us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:36:36.330385 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:36:36.330401 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:36:36.330902 (+ 501us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:36:36.331376 (+ 474us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:36:36.331380 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:36:36.331382 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:36:36.331578 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:36:36.332332 (+ 754us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:36:36.332339 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:36:36.332713 (+ 374us) client_negotiation.cc:769] Sending connection context
0810 11:36:36.332981 (+ 268us) client_negotiation.cc:240] Negotiation successful
0810 11:36:36.333485 (+ 504us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":643,"thread_start_us":366,"threads_started":1}
I20260810 11:36:36.336616 31901 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.18.253" port: 46623 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.8.18.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.18.252@KRBTEST.COM" created.
Aug 10 11:36:36 dist-test-slave-srgv krb5kdc[31741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361796, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.253@KRBTEST.COM for kudu/127.8.18.253@KRBTEST.COM
I20260810 11:36:36.352257 31904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:36:36.337575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.253:46623 (local address 127.0.0.1:57954)
0810 11:36:36.337790 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0810 11:36:36.337817 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0810 11:36:36.337959 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:36:36.338987 (+ 1028us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:36:36.338991 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:36:36.339467 (+ 476us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:36:36.339620 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:36:36.339630 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:36:36.341175 (+ 1545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:36:36.341183 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:36:36.342205 (+ 1022us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:36:36.342218 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:36:36.343144 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:36:36.343150 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:36:36.343326 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:36:36.343979 (+ 653us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:36:36.343998 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:36:36.346188 (+ 2190us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:36:36.349604 (+ 3416us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:36:36.349619 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:36:36.349623 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:36:36.350002 (+ 379us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:36:36.350394 (+ 392us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:36:36.350400 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:36:36.350403 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:36:36.350545 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:36:36.351192 (+ 647us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:36:36.351197 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:36:36.351344 (+ 147us) client_negotiation.cc:769] Sending connection context
0810 11:36:36.351481 (+ 137us) client_negotiation.cc:240] Negotiation successful
0810 11:36:36.351801 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":68}
I20260810 11:36:36.352339 31906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:36:36.337583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57954 (local address 127.8.18.253:46623)
0810 11:36:36.338560 (+ 977us) server_negotiation.cc:206] Beginning negotiation
0810 11:36:36.338566 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:36:36.338592 (+ 26us) server_negotiation.cc:407] Connection header received
0810 11:36:36.338638 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:36:36.338644 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:36:36.338833 (+ 189us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:36:36.338995 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:36:36.339808 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:36:36.341026 (+ 1218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:36:36.342346 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:36:36.342958 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:36:36.343129 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:36:36.346409 (+ 3280us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:36:36.346423 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:36:36.346439 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:36:36.346475 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:36:36.349469 (+ 2994us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:36:36.350158 (+ 689us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:36:36.350165 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:36:36.350167 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:36:36.350286 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:36:36.350670 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:36:36.350677 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:36:36.350684 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:36:36.351043 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:36:36.351270 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0810 11:36:36.351647 (+ 377us) server_negotiation.cc:299] Negotiation successful
0810 11:36:36.351924 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":36,"server-negotiator.queue_time_us":823,"thread_start_us":635,"threads_started":1}
I20260810 11:36:36.355162 31901 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.18.252" port: 45977 } has no permanent_uuid. Determining permanent_uuid...
I20260810 11:36:36.356511 31904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:36:36.355908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.252:45977 (local address 127.0.0.1:38518)
0810 11:36:36.356134 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0810 11:36:36.356297 (+ 163us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.252:45977: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
W20260810 11:36:36.356815 31849 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.18.252:45977: connect: Connection refused (error 111)
I20260810 11:36:36.358667 31904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:36:36.358187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.252:45977 (local address 127.0.0.1:38522)
0810 11:36:36.358384 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0810 11:36:36.358495 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.252:45977: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260810 11:36:36.358922 31901 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.18.252:45977: Network error: Client connection negotiation failed: client connection to 127.8.18.252:45977: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.18.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.252.keytab.
Entry for principal kudu/127.8.18.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.252.keytab.
I20260810 11:36:36.381325 8267 external_mini_cluster.cc:1326] Running /tmp/dist-test-task_r3aKW/build/asan/bin/kudu
/tmp/dist-test-task_r3aKW/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786361401922362-8267-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786361401922362-8267-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786361401922362-8267-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786361401922362-8267-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.18.252:45977
--webserver_interface=127.8.18.252
--webserver_port=0
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.252.keytab
--principal=kudu/127.8.18.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.18.212:37737
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.18.254:40371,127.8.18.253:46623,127.8.18.252:45977
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786361401922362-8267-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786361401922362-8267-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786361401922362-8267-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 11:36:36.402877 31826 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.18.253" port: 46623 } attempt: 5
I20260810 11:36:36.408350 31901 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.18.252" port: 45977 } attempt: 1
I20260810 11:36:36.409749 31904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:36:36.409075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.252:45977 (local address 127.0.0.1:38534)
0810 11:36:36.409309 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0810 11:36:36.409580 (+ 271us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.252:45977: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110}
I20260810 11:36:36.410970 31904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:36:36.410595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.252:45977 (local address 127.0.0.1:38542)
0810 11:36:36.410765 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0810 11:36:36.410852 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.252:45977: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260810 11:36:36.411266 31901 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.18.252:45977: Network error: Client connection negotiation failed: client connection to 127.8.18.252:45977: connect: Connection refused (error 111)
Aug 10 11:36:36 dist-test-slave-srgv krb5kdc[31741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361795, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.254@KRBTEST.COM for kudu/127.8.18.253@KRBTEST.COM
I20260810 11:36:36.419003 31906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:36:36.403846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57962 (local address 127.8.18.253:46623)
0810 11:36:36.404072 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0810 11:36:36.404078 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:36:36.404102 (+ 24us) server_negotiation.cc:407] Connection header received
0810 11:36:36.404354 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:36:36.404358 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:36:36.404426 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:36:36.404537 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:36:36.405429 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:36:36.406206 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:36:36.407492 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:36:36.407953 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:36:36.408109 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:36:36.413069 (+ 4960us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:36:36.413074 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:36:36.413078 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:36:36.413113 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:36:36.416033 (+ 2920us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:36:36.416834 (+ 801us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:36:36.416847 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:36:36.416850 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:36:36.416984 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:36:36.417431 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:36:36.417437 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:36:36.417446 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:36:36.417773 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:36:36.417944 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0810 11:36:36.418411 (+ 467us) server_negotiation.cc:299] Negotiation successful
0810 11:36:36.418646 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260810 11:36:36.419005 31830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:36:36.403731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.253:46623 (local address 127.0.0.1:57962)
0810 11:36:36.403980 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0810 11:36:36.404016 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0810 11:36:36.404165 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:36:36.404572 (+ 407us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:36:36.404577 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:36:36.405034 (+ 457us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:36:36.405235 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:36:36.405245 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:36:36.406369 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:36:36.406375 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:36:36.407304 (+ 929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:36:36.407317 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:36:36.408134 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:36:36.408140 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:36:36.408534 (+ 394us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:36:36.409667 (+ 1133us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:36:36.409707 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:36:36.412828 (+ 3121us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:36:36.416236 (+ 3408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:36:36.416247 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:36:36.416250 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:36:36.416684 (+ 434us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:36:36.417117 (+ 433us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:36:36.417122 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:36:36.417126 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:36:36.417284 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:36:36.417904 (+ 620us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:36:36.417909 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:36:36.418077 (+ 168us) client_negotiation.cc:769] Sending connection context
0810 11:36:36.418285 (+ 208us) client_negotiation.cc:240] Negotiation successful
0810 11:36:36.418638 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":84}
I20260810 11:36:36.420472 31826 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.18.252" port: 45977 } has no permanent_uuid. Determining permanent_uuid...
I20260810 11:36:36.421546 31830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:36:36.421085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.252:45977 (local address 127.0.0.1:38546)
0810 11:36:36.421253 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0810 11:36:36.421337 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.252:45977: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260810 11:36:36.422513 31830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:36:36.422170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.252:45977 (local address 127.0.0.1:38558)
0810 11:36:36.422318 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0810 11:36:36.422411 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.252:45977: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260810 11:36:36.422739 31826 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.18.252:45977: Network error: Client connection negotiation failed: client connection to 127.8.18.252:45977: connect: Connection refused (error 111)
I20260810 11:36:36.474030 31826 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.18.252" port: 45977 } attempt: 1
I20260810 11:36:36.475353 31830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:36:36.474751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.252:45977 (local address 127.0.0.1:38566)
0810 11:36:36.475009 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0810 11:36:36.475133 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.252:45977: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
I20260810 11:36:36.476444 31830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:36:36.476042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.252:45977 (local address 127.0.0.1:38574)
0810 11:36:36.476203 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0810 11:36:36.476291 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.252:45977: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260810 11:36:36.476715 31826 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.18.252:45977: Network error: Client connection negotiation failed: client connection to 127.8.18.252:45977: connect: Connection refused (error 111)
I20260810 11:36:36.479522 31901 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.18.252" port: 45977 } attempt: 2
I20260810 11:36:36.480574 31904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:36:36.480161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.252:45977 (local address 127.0.0.1:38590)
0810 11:36:36.480346 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0810 11:36:36.480431 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.252:45977: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260810 11:36:36.481600 31904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:36:36.481202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.252:45977 (local address 127.0.0.1:38594)
0810 11:36:36.481353 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0810 11:36:36.481486 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.252:45977: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260810 11:36:36.481817 31901 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.18.252:45977: Network error: Client connection negotiation failed: client connection to 127.8.18.252:45977: connect: Connection refused (error 111)
I20260810 11:36:36.544974 31826 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.18.252" port: 45977 } attempt: 2
I20260810 11:36:36.546326 31830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:36:36.545841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.252:45977 (local address 127.0.0.1:38596)
0810 11:36:36.546061 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0810 11:36:36.546154 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.252:45977: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260810 11:36:36.547478 31830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:36:36.547125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.252:45977 (local address 127.0.0.1:38598)
0810 11:36:36.547288 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0810 11:36:36.547370 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.252:45977: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260810 11:36:36.547709 31826 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.18.252:45977: Network error: Client connection negotiation failed: client connection to 127.8.18.252:45977: connect: Connection refused (error 111)
I20260810 11:36:36.573046 31901 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.18.252" port: 45977 } attempt: 3
I20260810 11:36:36.574335 31904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:36:36.573796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.252:45977 (local address 127.0.0.1:38612)
0810 11:36:36.574058 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0810 11:36:36.574150 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.252:45977: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260810 11:36:36.575495 31904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:36:36.575015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.252:45977 (local address 127.0.0.1:38626)
0810 11:36:36.575276 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0810 11:36:36.575373 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.252:45977: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143}
W20260810 11:36:36.575703 31901 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.18.252:45977: Network error: Client connection negotiation failed: client connection to 127.8.18.252:45977: connect: Connection refused (error 111)
I20260810 11:36:36.653932 31826 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.18.252" port: 45977 } attempt: 3
I20260810 11:36:36.655350 31830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:36:36.654717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.252:45977 (local address 127.0.0.1:38642)
0810 11:36:36.654958 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0810 11:36:36.655089 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.252:45977: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95}
I20260810 11:36:36.656656 31830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:36:36.656243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.252:45977 (local address 127.0.0.1:38650)
0810 11:36:36.656440 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0810 11:36:36.656536 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.252:45977: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260810 11:36:36.656874 31826 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.18.252:45977: Network error: Client connection negotiation failed: client connection to 127.8.18.252:45977: connect: Connection refused (error 111)
I20260810 11:36:36.718912 31901 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.18.252" port: 45977 } attempt: 4
I20260810 11:36:36.720512 31904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:36:36.719776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.252:45977 (local address 127.0.0.1:38656)
0810 11:36:36.720097 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0810 11:36:36.720264 (+ 167us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.252:45977: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260810 11:36:36.721899 31904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:36:36.721343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.252:45977 (local address 127.0.0.1:38662)
0810 11:36:36.721604 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0810 11:36:36.721718 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.252:45977: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101}
W20260810 11:36:36.722178 31901 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.18.252:45977: Network error: Client connection negotiation failed: client connection to 127.8.18.252:45977: connect: Connection refused (error 111)
W20260810 11:36:36.760356 31910 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260810 11:36:36.760908 31910 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 11:36:36.761073 31910 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 11:36:36.761247 31910 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 11:36:36.769272 31910 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 11:36:36.769398 31910 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 11:36:36.769542 31910 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 11:36:36.769640 31910 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 11:36:36.769742 31910 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 11:36:36.780622 31910 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.18.212:37737
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786361401922362-8267-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786361401922362-8267-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.8.18.254:40371,127.8.18.253:46623,127.8.18.252:45977
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786361401922362-8267-0/krb5kdc/kudu_127.8.18.252.keytab
--principal=kudu/127.8.18.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.18.252:45977
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786361401922362-8267-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.8.18.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786361401922362-8267-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 10:32:47 UTC on ccycloud.quasar-nosthy.root.comops.site
build id 4577509
ASAN enabled
I20260810 11:36:36.782179 31910 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 11:36:36.783726 31910 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 11:36:36.796983 31920 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:36:36.796984 31916 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 11:36:36.797016 31917 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 11:36:36.798345 31910 server_base.cc:1034] running on GCE node
I20260810 11:36:36.799270 31910 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 11:36:36.800951 31910 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 11:36:36.802235 31910 hybrid_clock.cc:648] HybridClock initialized: now 1786361796802166 us; error 44 us; skew 500 ppm
I20260810 11:36:36.802641 31910 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 11:36:36 dist-test-slave-srgv krb5kdc[31741](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361796, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 11:36:36.806386 31910 init.cc:377] Logged in from keytab as kudu/127.8.18.252@KRBTEST.COM (short username kudu)
I20260810 11:36:36.808192 31910 webserver.cc:460] Webserver started at http://127.8.18.252:39941/ using document root <none> and password file <none>
I20260810 11:36:36.809013 31910 fs_manager.cc:359] Metadata directory not provided
I20260810 11:36:36.809171 31910 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 11:36:36.809618 31910 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 11:36:36.813054 31910 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786361401922362-8267-0/minicluster-data/master-2/data/instance:
uuid: "1c2c2aa8e1d14fa08577c0ddee655983"
format_stamp: "Formatted at 2026-08-10 11:36:36 on dist-test-slave-srgv"
server_key: "62a7dd2b519fe1626c1a764136a31220"
server_key_iv: "08a82637a6838f4bab67f7bb79c411fb"
server_key_version: "encryptionkey@0"
I20260810 11:36:36.814085 31910 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786361401922362-8267-0/minicluster-data/master-2/wal/instance:
uuid: "1c2c2aa8e1d14fa08577c0ddee655983"
format_stamp: "Formatted at 2026-08-10 11:36:36 on dist-test-slave-srgv"
server_key: "62a7dd2b519fe1626c1a764136a31220"
server_key_iv: "08a82637a6838f4bab67f7bb79c411fb"
server_key_version: "encryptionkey@0"
I20260810 11:36:36.820920 31910 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260810 11:36:36.825495 31927 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 11:36:36.827132 31910 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260810 11:36:36.827418 31910 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786361401922362-8267-0/minicluster-data/master-2/data,/tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786361401922362-8267-0/minicluster-data/master-2/wal
uuid: "1c2c2aa8e1d14fa08577c0ddee655983"
format_stamp: "Formatted at 2026-08-10 11:36:36 on dist-test-slave-srgv"
server_key: "62a7dd2b519fe1626c1a764136a31220"
server_key_iv: "08a82637a6838f4bab67f7bb79c411fb"
server_key_version: "encryptionkey@0"
I20260810 11:36:36.827718 31910 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786361401922362-8267-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786361401922362-8267-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786361401922362-8267-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 11:36:36.834161 31826 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.18.252" port: 45977 } attempt: 4
I20260810 11:36:36.835489 31830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:36:36.835055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.252:45977 (local address 127.0.0.1:38666)
0810 11:36:36.835247 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0810 11:36:36.835338 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.252:45977: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260810 11:36:36.836587 31830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:36:36.836237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.252:45977 (local address 127.0.0.1:38668)
0810 11:36:36.836384 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0810 11:36:36.836461 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.18.252:45977: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260810 11:36:36.836846 31826 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.18.252:45977: Network error: Client connection negotiation failed: client connection to 127.8.18.252:45977: connect: Connection refused (error 111)
I20260810 11:36:36.869541 31910 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 11:36:36.874285 31910 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 11:36:36.874629 31910 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 11:36:36.894902 31910 rpc_server.cc:307] RPC server started. Bound to: 127.8.18.252:45977
I20260810 11:36:36.894922 31980 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.18.252:45977 every 8 connection(s)
I20260810 11:36:36.896437 31910 server_base.cc:1166] Dumped server information to /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786361401922362-8267-0/minicluster-data/master-2/data/info.pb
I20260810 11:36:36.897257 8267 external_mini_cluster.cc:1388] Started /tmp/dist-test-task_r3aKW/build/asan/bin/kudu as pid 31910
I20260810 11:36:36.897481 8267 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task_r3aKW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786361401922362-8267-0/minicluster-data/master-2/wal/instance
I20260810 11:36:36.898097 8267 external_mini_cluster.cc:1428] Setting key 488df7017bb5cb4846305c6b1c89380a
I20260810 11:36:36.901293 31981 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 11:36:36.910971 31981 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.18.254" port: 40371 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 11:36:36 dist-test-slave-srgv krb5kdc[31741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361795, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.18.252@KRBTEST.COM
Aug 10 11:36:36 dist-test-slave-srgv krb5kdc[31741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361796, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.252@KRBTEST.COM for kudu/127.8.18.254@KRBTEST.COM
I20260810 11:36:36.926208 31984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:36:36.900171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38684 (local address 127.8.18.252:45977)
0810 11:36:36.901013 (+ 842us) server_negotiation.cc:206] Beginning negotiation
0810 11:36:36.901025 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 11:36:36.901066 (+ 41us) server_negotiation.cc:407] Connection header received
0810 11:36:36.902793 (+ 1727us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:36:36.902816 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:36:36.903237 (+ 421us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:36:36.903576 (+ 339us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:36:36.905683 (+ 2107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:36:36.907287 (+ 1604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:36:36.909571 (+ 2284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:36:36.910392 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:36:36.910577 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:36:36.916580 (+ 6003us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:36:36.916593 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:36:36.916614 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:36:36.916663 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:36:36.920979 (+ 4316us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:36:36.921960 (+ 981us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:36:36.921968 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:36:36.921974 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:36:36.922169 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:36:36.922726 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:36:36.922733 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:36:36.922736 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:36:36.923665 (+ 929us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:36:36.924038 (+ 373us) server_negotiation.cc:1035] Waiting for connection context
0810 11:36:36.924477 (+ 439us) server_negotiation.cc:299] Negotiation successful
0810 11:36:36.924983 (+ 506us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":610,"spinlock_wait_cycles":35584,"thread_start_us":374,"threads_started":1}
I20260810 11:36:36.933396 31985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:36:36.914988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51396 (local address 127.8.18.254:40371)
0810 11:36:36.915721 (+ 733us) server_negotiation.cc:206] Beginning negotiation
0810 11:36:36.915728 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:36:36.916435 (+ 707us) server_negotiation.cc:407] Connection header received
0810 11:36:36.916637 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:36:36.916641 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:36:36.916715 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:36:36.916805 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:36:36.918714 (+ 1909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:36:36.919615 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:36:36.921703 (+ 2088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:36:36.922254 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:36:36.922422 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:36:36.926348 (+ 3926us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:36:36.926354 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:36:36.926358 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:36:36.926400 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:36:36.929804 (+ 3404us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:36:36.930830 (+ 1026us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:36:36.930836 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:36:36.930839 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:36:36.931049 (+ 210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:36:36.931486 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:36:36.931493 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:36:36.931497 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:36:36.932009 (+ 512us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:36:36.932192 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0810 11:36:36.932870 (+ 678us) server_negotiation.cc:299] Negotiation successful
0810 11:36:36.933065 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":538,"thread_start_us":386,"threads_started":1}
I20260810 11:36:36.933598 31986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:36:36.915121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.254:40371 (local address 127.0.0.1:51396)
0810 11:36:36.916212 (+ 1091us) negotiation.cc:107] Waiting for socket to connect
0810 11:36:36.916256 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 11:36:36.916468 (+ 212us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:36:36.916907 (+ 439us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:36:36.916915 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:36:36.918051 (+ 1136us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:36:36.918512 (+ 461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:36:36.918530 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:36:36.919799 (+ 1269us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:36:36.919805 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:36:36.921537 (+ 1732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:36:36.921550 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:36:36.922465 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:36:36.922470 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:36:36.922602 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:36:36.923396 (+ 794us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:36:36.923419 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:36:36.926109 (+ 2690us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:36:36.929950 (+ 3841us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:36:36.929964 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:36:36.929972 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:36:36.930665 (+ 693us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:36:36.931189 (+ 524us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:36:36.931193 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:36:36.931195 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:36:36.931336 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:36:36.932286 (+ 950us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:36:36.932292 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:36:36.932450 (+ 158us) client_negotiation.cc:769] Sending connection context
0810 11:36:36.932664 (+ 214us) client_negotiation.cc:240] Negotiation successful
0810 11:36:36.933095 (+ 431us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":753,"thread_start_us":535,"threads_started":1}
I20260810 11:36:36.935180 31981 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.18.253" port: 46623 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 11:36:36 dist-test-slave-srgv krb5kdc[31741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361795, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.18.254@KRBTEST.COM
Aug 10 11:36:36 dist-test-slave-srgv krb5kdc[31741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361796, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.252@KRBTEST.COM for kudu/127.8.18.253@KRBTEST.COM
I20260810 11:36:36.948683 31987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:36:36.929803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51410 (local address 127.8.18.254:40371)
0810 11:36:36.930562 (+ 759us) server_negotiation.cc:206] Beginning negotiation
0810 11:36:36.930570 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:36:36.930599 (+ 29us) server_negotiation.cc:407] Connection header received
0810 11:36:36.930652 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:36:36.930656 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:36:36.930740 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:36:36.930842 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:36:36.932643 (+ 1801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:36:36.934038 (+ 1395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:36:36.935468 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:36:36.936138 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:36:36.936290 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:36:36.941625 (+ 5335us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:36:36.941654 (+ 29us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:36:36.941659 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:36:36.941711 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:36:36.945104 (+ 3393us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:36:36.946014 (+ 910us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:36:36.946020 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:36:36.946024 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:36:36.946236 (+ 212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:36:36.946719 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:36:36.946727 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:36:36.946733 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:36:36.947159 (+ 426us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:36:36.947462 (+ 303us) server_negotiation.cc:1035] Waiting for connection context
0810 11:36:36.947933 (+ 471us) server_negotiation.cc:299] Negotiation successful
0810 11:36:36.948225 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":584,"thread_start_us":352,"threads_started":1}
I20260810 11:36:36.952309 31988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:36:36.936042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57966 (local address 127.8.18.253:46623)
0810 11:36:36.936857 (+ 815us) server_negotiation.cc:206] Beginning negotiation
0810 11:36:36.936863 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:36:36.936882 (+ 19us) server_negotiation.cc:407] Connection header received
0810 11:36:36.936943 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:36:36.936947 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:36:36.937009 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:36:36.937152 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:36:36.938108 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:36:36.939348 (+ 1240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:36:36.940745 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:36:36.941481 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:36:36.941587 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:36:36.945485 (+ 3898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:36:36.945496 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:36:36.945505 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:36:36.945546 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:36:36.949066 (+ 3520us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:36:36.949955 (+ 889us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:36:36.949968 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:36:36.949971 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:36:36.950120 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:36:36.950583 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:36:36.950588 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:36:36.950593 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:36:36.951019 (+ 426us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:36:36.951209 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0810 11:36:36.951705 (+ 496us) server_negotiation.cc:299] Negotiation successful
0810 11:36:36.951998 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":669,"thread_start_us":534,"threads_started":1}
I20260810 11:36:36.952324 31986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:36:36.935919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.253:46623 (local address 127.0.0.1:57966)
0810 11:36:36.936117 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0810 11:36:36.936138 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 11:36:36.936260 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:36:36.937237 (+ 977us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:36:36.937240 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:36:36.937719 (+ 479us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:36:36.937886 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:36:36.937895 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:36:36.939524 (+ 1629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:36:36.939528 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:36:36.940567 (+ 1039us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:36:36.940582 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:36:36.941644 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:36:36.941651 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:36:36.941868 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:36:36.942529 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:36:36.942562 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:36:36.945158 (+ 2596us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:36:36.949245 (+ 4087us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:36:36.949251 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:36:36.949255 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:36:36.949808 (+ 553us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:36:36.950258 (+ 450us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:36:36.950264 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:36:36.950268 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:36:36.950454 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:36:36.951292 (+ 838us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:36:36.951296 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:36:36.951446 (+ 150us) client_negotiation.cc:769] Sending connection context
0810 11:36:36.951568 (+ 122us) client_negotiation.cc:240] Negotiation successful
0810 11:36:36.951924 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":69}
I20260810 11:36:36.953591 31981 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.18.252" port: 45977 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 11:36:36 dist-test-slave-srgv krb5kdc[31741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361795, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.18.253@KRBTEST.COM
Aug 10 11:36:36 dist-test-slave-srgv krb5kdc[31741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361796, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.252@KRBTEST.COM for kudu/127.8.18.252@KRBTEST.COM
I20260810 11:36:36.971078 31986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:36:36.954425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.252:45977 (local address 127.0.0.1:38686)
0810 11:36:36.954608 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0810 11:36:36.954634 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0810 11:36:36.954761 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:36:36.955121 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:36:36.955126 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:36:36.955706 (+ 580us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:36:36.955886 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:36:36.955898 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:36:36.957239 (+ 1341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:36:36.957244 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:36:36.958641 (+ 1397us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:36:36.958662 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:36:36.959480 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:36:36.959490 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:36:36.959757 (+ 267us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:36:36.960398 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:36:36.960432 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:36:36.962937 (+ 2505us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:36:36.967655 (+ 4718us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:36:36.967661 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:36:36.967664 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:36:36.968266 (+ 602us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:36:36.968809 (+ 543us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:36:36.968815 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:36:36.968819 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:36:36.968999 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:36:36.969845 (+ 846us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:36:36.969850 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:36:36.970042 (+ 192us) client_negotiation.cc:769] Sending connection context
0810 11:36:36.970241 (+ 199us) client_negotiation.cc:240] Negotiation successful
0810 11:36:36.970612 (+ 371us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260810 11:36:36.971117 31989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:36:36.950789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57972 (local address 127.8.18.253:46623)
0810 11:36:36.951434 (+ 645us) server_negotiation.cc:206] Beginning negotiation
0810 11:36:36.951438 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 11:36:36.951457 (+ 19us) server_negotiation.cc:407] Connection header received
0810 11:36:36.951497 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:36:36.951501 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:36:36.951589 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:36:36.951701 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:36:36.953507 (+ 1806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:36:36.954567 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:36:36.956370 (+ 1803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:36:36.957081 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:36:36.957242 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:36:36.963071 (+ 5829us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:36:36.963079 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:36:36.963084 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:36:36.963136 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:36:36.967480 (+ 4344us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:36:36.968458 (+ 978us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:36:36.968463 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:36:36.968466 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:36:36.968671 (+ 205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:36:36.969186 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:36:36.969192 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:36:36.969196 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:36:36.969646 (+ 450us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:36:36.969928 (+ 282us) server_negotiation.cc:1035] Waiting for connection context
0810 11:36:36.970371 (+ 443us) server_negotiation.cc:299] Negotiation successful
0810 11:36:36.970630 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":542,"thread_start_us":319,"threads_started":1}
I20260810 11:36:36.971312 31984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:36:36.954620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38686 (local address 127.8.18.252:45977)
0810 11:36:36.954855 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0810 11:36:36.954859 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 11:36:36.954874 (+ 15us) server_negotiation.cc:407] Connection header received
0810 11:36:36.954915 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:36:36.954919 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:36:36.954974 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:36:36.955055 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:36:36.956034 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:36:36.957081 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:36:36.958810 (+ 1729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:36:36.959280 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:36:36.959385 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:36:36.963158 (+ 3773us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:36:36.963167 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:36:36.963171 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:36:36.963213 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:36:36.967373 (+ 4160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:36:36.968456 (+ 1083us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:36:36.968462 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:36:36.968465 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:36:36.968664 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:36:36.969186 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:36:36.969192 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:36:36.969196 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:36:36.969647 (+ 451us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:36:36.969811 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0810 11:36:36.970371 (+ 560us) server_negotiation.cc:299] Negotiation successful
0810 11:36:36.970828 (+ 457us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":165,"server-negotiator.queue_time_us":132}
I20260810 11:36:36.976436 8267 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 11:36:36.976986 31981 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1c2c2aa8e1d14fa08577c0ddee655983: Bootstrap starting.
I20260810 11:36:36.980827 31981 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1c2c2aa8e1d14fa08577c0ddee655983: Neither blocks nor log segments found. Creating new log.
I20260810 11:36:36.982127 31981 log.cc:824] T 00000000000000000000000000000000 P 1c2c2aa8e1d14fa08577c0ddee655983: Log is configured to *not* fsync() on all Append() calls
I20260810 11:36:36.979724 31799 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "72340a6f145b46d28372cd7526703a45"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51410
I20260810 11:36:36.985703 31981 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1c2c2aa8e1d14fa08577c0ddee655983: No bootstrap required, opened a new log
I20260810 11:36:36.993752 31981 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1c2c2aa8e1d14fa08577c0ddee655983 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72340a6f145b46d28372cd7526703a45" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 40371 } } peers { permanent_uuid: "7a1346fc8626434aae9378df2825aecf" member_type: VOTER last_known_addr { host: "127.8.18.253" port: 46623 } } peers { permanent_uuid: "1c2c2aa8e1d14fa08577c0ddee655983" member_type: VOTER last_known_addr { host: "127.8.18.252" port: 45977 } }
I20260810 11:36:36.994192 31981 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1c2c2aa8e1d14fa08577c0ddee655983 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c2c2aa8e1d14fa08577c0ddee655983, State: Initialized, Role: FOLLOWER
I20260810 11:36:36.994696 31981 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1c2c2aa8e1d14fa08577c0ddee655983 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72340a6f145b46d28372cd7526703a45" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 40371 } } peers { permanent_uuid: "7a1346fc8626434aae9378df2825aecf" member_type: VOTER last_known_addr { host: "127.8.18.253" port: 46623 } } peers { permanent_uuid: "1c2c2aa8e1d14fa08577c0ddee655983" member_type: VOTER last_known_addr { host: "127.8.18.252" port: 45977 } }
I20260810 11:36:36.995792 31991 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c2c2aa8e1d14fa08577c0ddee655983 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72340a6f145b46d28372cd7526703a45" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 40371 } } peers { permanent_uuid: "7a1346fc8626434aae9378df2825aecf" member_type: VOTER last_known_addr { host: "127.8.18.253" port: 46623 } } peers { permanent_uuid: "1c2c2aa8e1d14fa08577c0ddee655983" member_type: VOTER last_known_addr { host: "127.8.18.252" port: 45977 } } }
I20260810 11:36:36.996206 31991 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c2c2aa8e1d14fa08577c0ddee655983 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 11:36:36.996855 31981 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1c2c2aa8e1d14fa08577c0ddee655983 [sys.catalog]: configured and running, proceeding with master startup.
W20260810 11:36:37.009120 32002 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1c2c2aa8e1d14fa08577c0ddee655983: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 11:36:37.009346 32002 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 11:36:37.006702 31910 master_runner.cc:186] Error getting master registration for 127.8.18.254:40371: OK, instance_id { permanent_uuid: "72340a6f145b46d28372cd7526703a45" instance_seqno: 1786361795685460 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260810 11:36:37.019088 31910 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260810 11:36:37.021464 31901 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.18.252" port: 45977 } attempt: 5
Aug 10 11:36:37 dist-test-slave-srgv krb5kdc[31741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361796, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.253@KRBTEST.COM for kudu/127.8.18.252@KRBTEST.COM
I20260810 11:36:37.039639 31904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:36:37.022245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.252:45977 (local address 127.0.0.1:38700)
0810 11:36:37.022500 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0810 11:36:37.022534 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0810 11:36:37.022676 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:36:37.023185 (+ 509us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:36:37.023191 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:36:37.023783 (+ 592us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:36:37.024017 (+ 234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:36:37.024033 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:36:37.025572 (+ 1539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:36:37.025580 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:36:37.027056 (+ 1476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:36:37.027073 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:36:37.028186 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:36:37.028194 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:36:37.028467 (+ 273us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:36:37.029230 (+ 763us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:36:37.029267 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:36:37.032429 (+ 3162us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:36:37.036253 (+ 3824us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:36:37.036260 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:36:37.036264 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:36:37.036925 (+ 661us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:36:37.037458 (+ 533us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:36:37.037466 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:36:37.037471 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:36:37.037667 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:36:37.038410 (+ 743us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:36:37.038415 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:36:37.038618 (+ 203us) client_negotiation.cc:769] Sending connection context
0810 11:36:37.038840 (+ 222us) client_negotiation.cc:240] Negotiation successful
0810 11:36:37.039201 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":89}
I20260810 11:36:37.039722 31984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:36:37.022541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38700 (local address 127.8.18.252:45977)
0810 11:36:37.022772 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0810 11:36:37.022779 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:36:37.022806 (+ 27us) server_negotiation.cc:407] Connection header received
0810 11:36:37.022905 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:36:37.022912 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:36:37.023009 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:36:37.023123 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:36:37.024203 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:36:37.025377 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:36:37.027234 (+ 1857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:36:37.028007 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:36:37.028162 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:36:37.032752 (+ 4590us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:36:37.032761 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:36:37.032766 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:36:37.032815 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:36:37.036069 (+ 3254us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:36:37.037115 (+ 1046us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:36:37.037120 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:36:37.037122 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:36:37.037274 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:36:37.037828 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:36:37.037832 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:36:37.037834 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:36:37.038272 (+ 438us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:36:37.038405 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0810 11:36:37.039047 (+ 642us) server_negotiation.cc:299] Negotiation successful
0810 11:36:37.039332 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260810 11:36:37.046304 31901 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7a1346fc8626434aae9378df2825aecf: Bootstrap starting.
I20260810 11:36:37.049937 31901 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7a1346fc8626434aae9378df2825aecf: Neither blocks nor log segments found. Creating new log.
I20260810 11:36:37.051085 31901 log.cc:824] T 00000000000000000000000000000000 P 7a1346fc8626434aae9378df2825aecf: Log is configured to *not* fsync() on all Append() calls
I20260810 11:36:37.054509 31901 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7a1346fc8626434aae9378df2825aecf: No bootstrap required, opened a new log
I20260810 11:36:37.062258 31901 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7a1346fc8626434aae9378df2825aecf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72340a6f145b46d28372cd7526703a45" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 40371 } } peers { permanent_uuid: "7a1346fc8626434aae9378df2825aecf" member_type: VOTER last_known_addr { host: "127.8.18.253" port: 46623 } } peers { permanent_uuid: "1c2c2aa8e1d14fa08577c0ddee655983" member_type: VOTER last_known_addr { host: "127.8.18.252" port: 45977 } }
I20260810 11:36:37.062656 31901 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7a1346fc8626434aae9378df2825aecf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a1346fc8626434aae9378df2825aecf, State: Initialized, Role: FOLLOWER
I20260810 11:36:37.063169 31901 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7a1346fc8626434aae9378df2825aecf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72340a6f145b46d28372cd7526703a45" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 40371 } } peers { permanent_uuid: "7a1346fc8626434aae9378df2825aecf" member_type: VOTER last_known_addr { host: "127.8.18.253" port: 46623 } } peers { permanent_uuid: "1c2c2aa8e1d14fa08577c0ddee655983" member_type: VOTER last_known_addr { host: "127.8.18.252" port: 45977 } }
I20260810 11:36:37.064284 32005 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7a1346fc8626434aae9378df2825aecf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72340a6f145b46d28372cd7526703a45" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 40371 } } peers { permanent_uuid: "7a1346fc8626434aae9378df2825aecf" member_type: VOTER last_known_addr { host: "127.8.18.253" port: 46623 } } peers { permanent_uuid: "1c2c2aa8e1d14fa08577c0ddee655983" member_type: VOTER last_known_addr { host: "127.8.18.252" port: 45977 } } }
I20260810 11:36:37.064718 32005 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7a1346fc8626434aae9378df2825aecf [sys.catalog]: This master's current role is: FOLLOWER
I20260810 11:36:37.065706 31901 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7a1346fc8626434aae9378df2825aecf [sys.catalog]: configured and running, proceeding with master startup.
W20260810 11:36:37.076697 32016 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7a1346fc8626434aae9378df2825aecf: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 11:36:37.077673 32016 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 11:36:37.073621 31833 master_runner.cc:186] Error getting master registration for 127.8.18.254:40371: OK, instance_id { permanent_uuid: "72340a6f145b46d28372cd7526703a45" instance_seqno: 1786361795685460 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260810 11:36:37.085592 31833 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260810 11:36:37.114092 31826 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.18.252" port: 45977 } attempt: 5
Aug 10 11:36:37 dist-test-slave-srgv krb5kdc[31741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786361795, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.18.254@KRBTEST.COM for kudu/127.8.18.252@KRBTEST.COM
W20260810 11:36:37.126977 31934 tablet.cc:2369] T 00000000000000000000000000000000 P 1c2c2aa8e1d14fa08577c0ddee655983: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 11:36:37.134706 31830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:36:37.114888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.18.252:45977 (local address 127.0.0.1:38704)
0810 11:36:37.115200 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0810 11:36:37.115221 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 11:36:37.115366 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 11:36:37.115845 (+ 479us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 11:36:37.115851 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 11:36:37.116622 (+ 771us) client_negotiation.cc:189] Negotiated authn=SASL
0810 11:36:37.116861 (+ 239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:36:37.116875 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:36:37.118479 (+ 1604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:36:37.118484 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:36:37.119999 (+ 1515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 11:36:37.120028 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 11:36:37.121442 (+ 1414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 11:36:37.121449 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 11:36:37.121664 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:36:37.122467 (+ 803us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 11:36:37.122495 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0810 11:36:37.125799 (+ 3304us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 11:36:37.130980 (+ 5181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:36:37.130988 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:36:37.130995 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:36:37.131713 (+ 718us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:36:37.132385 (+ 672us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 11:36:37.132391 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 11:36:37.132395 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 11:36:37.132627 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 11:36:37.133527 (+ 900us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 11:36:37.133533 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 11:36:37.133733 (+ 200us) client_negotiation.cc:769] Sending connection context
0810 11:36:37.133898 (+ 165us) client_negotiation.cc:240] Negotiation successful
0810 11:36:37.134248 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175}
I20260810 11:36:37.135815 31984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:36:37.114983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38704 (local address 127.8.18.252:45977)
0810 11:36:37.115206 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0810 11:36:37.115212 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:36:37.115290 (+ 78us) server_negotiation.cc:407] Connection header received
0810 11:36:37.115510 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:36:37.115515 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:36:37.115637 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:36:37.115771 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:36:37.117058 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:36:37.118278 (+ 1220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:36:37.120237 (+ 1959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:36:37.121008 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:36:37.121166 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:36:37.126115 (+ 4949us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:36:37.126124 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:36:37.126128 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:36:37.126178 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:36:37.130635 (+ 4457us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:36:37.131914 (+ 1279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:36:37.131921 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:36:37.131925 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:36:37.132137 (+ 212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:36:37.132785 (+ 648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:36:37.132791 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:36:37.132795 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:36:37.133263 (+ 468us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:36:37.133452 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0810 11:36:37.135135 (+ 1683us) server_negotiation.cc:299] Negotiation successful
0810 11:36:37.135431 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260810 11:36:37.142251 31826 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 72340a6f145b46d28372cd7526703a45: Bootstrap starting.
I20260810 11:36:37.146917 31826 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 72340a6f145b46d28372cd7526703a45: Neither blocks nor log segments found. Creating new log.
I20260810 11:36:37.148478 31826 log.cc:824] T 00000000000000000000000000000000 P 72340a6f145b46d28372cd7526703a45: Log is configured to *not* fsync() on all Append() calls
I20260810 11:36:37.152632 31826 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 72340a6f145b46d28372cd7526703a45: No bootstrap required, opened a new log
I20260810 11:36:37.161255 31826 raft_consensus.cc:348] T 00000000000000000000000000000000 P 72340a6f145b46d28372cd7526703a45 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72340a6f145b46d28372cd7526703a45" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 40371 } } peers { permanent_uuid: "7a1346fc8626434aae9378df2825aecf" member_type: VOTER last_known_addr { host: "127.8.18.253" port: 46623 } } peers { permanent_uuid: "1c2c2aa8e1d14fa08577c0ddee655983" member_type: VOTER last_known_addr { host: "127.8.18.252" port: 45977 } }
I20260810 11:36:37.161754 31826 raft_consensus.cc:729] T 00000000000000000000000000000000 P 72340a6f145b46d28372cd7526703a45 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72340a6f145b46d28372cd7526703a45, State: Initialized, Role: FOLLOWER
I20260810 11:36:37.162330 31826 consensus_queue.cc:260] T 00000000000000000000000000000000 P 72340a6f145b46d28372cd7526703a45 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72340a6f145b46d28372cd7526703a45" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 40371 } } peers { permanent_uuid: "7a1346fc8626434aae9378df2825aecf" member_type: VOTER last_known_addr { host: "127.8.18.253" port: 46623 } } peers { permanent_uuid: "1c2c2aa8e1d14fa08577c0ddee655983" member_type: VOTER last_known_addr { host: "127.8.18.252" port: 45977 } }
I20260810 11:36:37.163738 32020 sys_catalog.cc:455] T 00000000000000000000000000000000 P 72340a6f145b46d28372cd7526703a45 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72340a6f145b46d28372cd7526703a45" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 40371 } } peers { permanent_uuid: "7a1346fc8626434aae9378df2825aecf" member_type: VOTER last_known_addr { host: "127.8.18.253" port: 46623 } } peers { permanent_uuid: "1c2c2aa8e1d14fa08577c0ddee655983" member_type: VOTER last_known_addr { host: "127.8.18.252" port: 45977 } } }
I20260810 11:36:37.164579 32020 sys_catalog.cc:458] T 00000000000000000000000000000000 P 72340a6f145b46d28372cd7526703a45 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 11:36:37.165001 31826 sys_catalog.cc:564] T 00000000000000000000000000000000 P 72340a6f145b46d28372cd7526703a45 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 11:36:37.165083 31799 raft_consensus.cc:482] T 00000000000000000000000000000000 P 72340a6f145b46d28372cd7526703a45 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260810 11:36:37.165300 31799 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 72340a6f145b46d28372cd7526703a45 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:36:37.169006 31799 raft_consensus.cc:504] T 00000000000000000000000000000000 P 72340a6f145b46d28372cd7526703a45 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72340a6f145b46d28372cd7526703a45" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 40371 } } peers { permanent_uuid: "7a1346fc8626434aae9378df2825aecf" member_type: VOTER last_known_addr { host: "127.8.18.253" port: 46623 } } peers { permanent_uuid: "1c2c2aa8e1d14fa08577c0ddee655983" member_type: VOTER last_known_addr { host: "127.8.18.252" port: 45977 } }
I20260810 11:36:37.171926 31875 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "72340a6f145b46d28372cd7526703a45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "7a1346fc8626434aae9378df2825aecf"
I20260810 11:36:37.172533 31875 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7a1346fc8626434aae9378df2825aecf [term 0 FOLLOWER]: Advancing to term 1
W20260810 11:36:37.173880 31777 tablet.cc:2369] T 00000000000000000000000000000000 P 72340a6f145b46d28372cd7526703a45: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 11:36:37.176367 31875 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 7a1346fc8626434aae9378df2825aecf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 72340a6f145b46d28372cd7526703a45 in term 1.
I20260810 11:36:37.177439 31772 leader_election.cc:302] T 00000000000000000000000000000000 P 72340a6f145b46d28372cd7526703a45 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 72340a6f145b46d28372cd7526703a45, 7a1346fc8626434aae9378df2825aecf; no voters:
I20260810 11:36:37.179240 31956 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "72340a6f145b46d28372cd7526703a45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "1c2c2aa8e1d14fa08577c0ddee655983"
I20260810 11:36:37.179829 31956 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1c2c2aa8e1d14fa08577c0ddee655983 [term 0 FOLLOWER]: Advancing to term 1
I20260810 11:36:37.183717 31956 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1c2c2aa8e1d14fa08577c0ddee655983 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 72340a6f145b46d28372cd7526703a45 in term 1.
I20260810 11:36:37.184825 31799 leader_election.cc:288] T 00000000000000000000000000000000 P 72340a6f145b46d28372cd7526703a45 [CANDIDATE]: Term 1 election: Requested vote from peers 7a1346fc8626434aae9378df2825aecf (127.8.18.253:46623), 1c2c2aa8e1d14fa08577c0ddee655983 (127.8.18.252:45977)
I20260810 11:36:37.192097 32020 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 72340a6f145b46d28372cd7526703a45 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 11:36:37.201962 32020 raft_consensus.cc:686] T 00000000000000000000000000000000 P 72340a6f145b46d28372cd7526703a45 [term 1 LEADER]: Becoming Leader. State: Replica: 72340a6f145b46d28372cd7526703a45, State: Running, Role: LEADER
I20260810 11:36:37.203102 32020 consensus_queue.cc:237] T 00000000000000000000000000000000 P 72340a6f145b46d28372cd7526703a45 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72340a6f145b46d28372cd7526703a45" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 40371 } } peers { permanent_uuid: "7a1346fc8626434aae9378df2825aecf" member_type: VOTER last_known_addr { host: "127.8.18.253" port: 46623 } } peers { permanent_uuid: "1c2c2aa8e1d14fa08577c0ddee655983" member_type: VOTER last_known_addr { host: "127.8.18.252" port: 45977 } }
I20260810 11:36:37.214982 32028 sys_catalog.cc:455] T 00000000000000000000000000000000 P 72340a6f145b46d28372cd7526703a45 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 72340a6f145b46d28372cd7526703a45. Latest consensus state: current_term: 1 leader_uuid: "72340a6f145b46d28372cd7526703a45" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72340a6f145b46d28372cd7526703a45" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 40371 } } peers { permanent_uuid: "7a1346fc8626434aae9378df2825aecf" member_type: VOTER last_known_addr { host: "127.8.18.253" port: 46623 } } peers { permanent_uuid: "1c2c2aa8e1d14fa08577c0ddee655983" member_type: VOTER last_known_addr { host: "127.8.18.252" port: 45977 } } }
I20260810 11:36:37.215688 32028 sys_catalog.cc:458] T 00000000000000000000000000000000 P 72340a6f145b46d28372cd7526703a45 [sys.catalog]: This master's current role is: LEADER
I20260810 11:36:37.221774 32035 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 11:36:37.255745 32035 catalog_manager.cc:1484] Initializing Kudu cluster ID...
W20260810 11:36:37.273511 31854 tablet.cc:2369] T 00000000000000000000000000000000 P 7a1346fc8626434aae9378df2825aecf: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 11:36:37.277992 32044 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260810 11:36:37.288690 31984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:36:37.252210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38712 (local address 127.8.18.252:45977)
0810 11:36:37.252417 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0810 11:36:37.252424 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:36:37.255970 (+ 3546us) server_negotiation.cc:407] Connection header received
0810 11:36:37.256276 (+ 306us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:36:37.256283 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:36:37.256378 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:36:37.256462 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:36:37.265245 (+ 8783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:36:37.266419 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:36:37.268582 (+ 2163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:36:37.269302 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:36:37.269468 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:36:37.273603 (+ 4135us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:36:37.273611 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:36:37.273617 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:36:37.273672 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:36:37.281320 (+ 7648us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:36:37.285599 (+ 4279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:36:37.285607 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:36:37.285612 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:36:37.285839 (+ 227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:36:37.286438 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:36:37.286444 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:36:37.286448 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:36:37.286986 (+ 538us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:36:37.287254 (+ 268us) server_negotiation.cc:1035] Waiting for connection context
0810 11:36:37.287954 (+ 700us) server_negotiation.cc:299] Negotiation successful
0810 11:36:37.288275 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260810 11:36:37.289835 31989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:36:37.239653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57976 (local address 127.8.18.253:46623)
0810 11:36:37.240042 (+ 389us) server_negotiation.cc:206] Beginning negotiation
0810 11:36:37.240052 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 11:36:37.251213 (+ 11161us) server_negotiation.cc:407] Connection header received
0810 11:36:37.251516 (+ 303us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:36:37.251523 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:36:37.251636 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:36:37.251763 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:36:37.259897 (+ 8134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:36:37.261098 (+ 1201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:36:37.265627 (+ 4529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:36:37.266376 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:36:37.266519 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:36:37.274896 (+ 8377us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:36:37.274904 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:36:37.274909 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:36:37.274957 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:36:37.279166 (+ 4209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:36:37.286953 (+ 7787us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:36:37.286959 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:36:37.286963 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:36:37.287152 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:36:37.287674 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:36:37.287690 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:36:37.287694 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:36:37.288116 (+ 422us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:36:37.288402 (+ 286us) server_negotiation.cc:1035] Waiting for connection context
0810 11:36:37.289159 (+ 757us) server_negotiation.cc:299] Negotiation successful
0810 11:36:37.289479 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":195}
I20260810 11:36:37.301654 31956 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1c2c2aa8e1d14fa08577c0ddee655983 [term 1 FOLLOWER]: Refusing update from remote peer 72340a6f145b46d28372cd7526703a45: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 11:36:37.303222 32028 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 72340a6f145b46d28372cd7526703a45 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c2c2aa8e1d14fa08577c0ddee655983" member_type: VOTER last_known_addr { host: "127.8.18.252" port: 45977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 11:36:37.314443 31987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:36:37.239309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51418 (local address 127.8.18.254:40371)
0810 11:36:37.239547 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0810 11:36:37.239555 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:36:37.255662 (+ 16107us) server_negotiation.cc:407] Connection header received
0810 11:36:37.260795 (+ 5133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:36:37.260801 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:36:37.260898 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:36:37.260978 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:36:37.266336 (+ 5358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:36:37.267446 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:36:37.278415 (+ 10969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:36:37.279241 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:36:37.279372 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:36:37.284413 (+ 5041us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 11:36:37.284421 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 11:36:37.284427 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 11:36:37.284473 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0810 11:36:37.296057 (+ 11584us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:36:37.307261 (+ 11204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:36:37.307269 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:36:37.307273 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:36:37.307518 (+ 245us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 11:36:37.308194 (+ 676us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 11:36:37.308200 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 11:36:37.308203 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 11:36:37.308682 (+ 479us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 11:36:37.309017 (+ 335us) server_negotiation.cc:1035] Waiting for connection context
0810 11:36:37.313654 (+ 4637us) server_negotiation.cc:299] Negotiation successful
0810 11:36:37.314027 (+ 373us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260810 11:36:37.322253 31875 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7a1346fc8626434aae9378df2825aecf [term 1 FOLLOWER]: Refusing update from remote peer 72340a6f145b46d28372cd7526703a45: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 11:36:37.323479 32028 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 72340a6f145b46d28372cd7526703a45 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a1346fc8626434aae9378df2825aecf" member_type: VOTER last_known_addr { host: "127.8.18.253" port: 46623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 11:36:37.352170 32005 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7a1346fc8626434aae9378df2825aecf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 72340a6f145b46d28372cd7526703a45. Latest consensus state: current_term: 1 leader_uuid: "72340a6f145b46d28372cd7526703a45" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72340a6f145b46d28372cd7526703a45" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 40371 } } peers { permanent_uuid: "7a1346fc8626434aae9378df2825aecf" member_type: VOTER last_known_addr { host: "127.8.18.253" port: 46623 } } peers { permanent_uuid: "1c2c2aa8e1d14fa08577c0ddee655983" member_type: VOTER last_known_addr { host: "127.8.18.252" port: 45977 } } }
I20260810 11:36:37.352538 32005 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7a1346fc8626434aae9378df2825aecf [sys.catalog]: This master's current role is: FOLLOWER
I20260810 11:36:37.356173 31991 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c2c2aa8e1d14fa08577c0ddee655983 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 72340a6f145b46d28372cd7526703a45. Latest consensus state: current_term: 1 leader_uuid: "72340a6f145b46d28372cd7526703a45" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72340a6f145b46d28372cd7526703a45" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 40371 } } peers { permanent_uuid: "7a1346fc8626434aae9378df2825aecf" member_type: VOTER last_known_addr { host: "127.8.18.253" port: 46623 } } peers { permanent_uuid: "1c2c2aa8e1d14fa08577c0ddee655983" member_type: VOTER last_known_addr { host: "127.8.18.252" port: 45977 } } }
I20260810 11:36:37.356526 31991 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c2c2aa8e1d14fa08577c0ddee655983 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 11:36:37.359313 32045 mvcc.cc:205] Tried to move back new op lower bound from 7316937921650094080 to 7316937921366138880. Current Snapshot: MvccSnapshot[applied={T|T < 7316937921650094080}]
I20260810 11:36:37.368923 32035 catalog_manager.cc:1347] Generated new cluster ID: ad4e5048dc7a4eeea9efe2407bf6adbb
I20260810 11:36:37.369076 32035 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 11:36:37.373390 32020 sys_catalog.cc:455] T 00000000000000000000000000000000 P 72340a6f145b46d28372cd7526703a45 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "72340a6f145b46d28372cd7526703a45" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72340a6f145b46d28372cd7526703a45" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 40371 } } peers { permanent_uuid: "7a1346fc8626434aae9378df2825aecf" member_type: VOTER last_known_addr { host: "127.8.18.253" port: 46623 } } peers { permanent_uuid: "1c2c2aa8e1d14fa08577c0ddee655983" member_type: VOTER last_known_addr { host: "127.8.18.252" port: 45977 } } }
I20260810 11:36:37.373854 32020 sys_catalog.cc:458] T 00000000000000000000000000000000 P 72340a6f145b46d28372cd7526703a45 [sys.catalog]: This master's current role is: LEADER
I20260810 11:36:37.374333 32028 sys_catalog.cc:455] T 00000000000000000000000000000000 P 72340a6f145b46d28372cd7526703a45 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "72340a6f145b46d28372cd7526703a45" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72340a6f145b46d28372cd7526703a45" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 40371 } } peers { permanent_uuid: "7a1346fc8626434aae9378df2825aecf" member_type: VOTER last_known_addr { host: "127.8.18.253" port: 46623 } } peers { permanent_uuid: "1c2c2aa8e1d14fa08577c0ddee655983" member_type: VOTER last_known_addr { host: "127.8.18.252" port: 45977 } } }
I20260810 11:36:37.374702 32028 sys_catalog.cc:458] T 00000000000000000000000000000000 P 72340a6f145b46d28372cd7526703a45 [sys.catalog]: This master's current role is: LEADER
I20260810 11:36:37.382692 32005 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7a1346fc8626434aae9378df2825aecf [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "72340a6f145b46d28372cd7526703a45" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72340a6f145b46d28372cd7526703a45" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 40371 } } peers { permanent_uuid: "7a1346fc8626434aae9378df2825aecf" member_type: VOTER last_known_addr { host: "127.8.18.253" port: 46623 } } peers { permanent_uuid: "1c2c2aa8e1d14fa08577c0ddee655983" member_type: VOTER last_known_addr { host: "127.8.18.252" port: 45977 } } }
I20260810 11:36:37.383041 32005 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7a1346fc8626434aae9378df2825aecf [sys.catalog]: This master's current role is: FOLLOWER
I20260810 11:36:37.390911 31991 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c2c2aa8e1d14fa08577c0ddee655983 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "72340a6f145b46d28372cd7526703a45" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72340a6f145b46d28372cd7526703a45" member_type: VOTER last_known_addr { host: "127.8.18.254" port: 40371 } } peers { permanent_uuid: "7a1346fc8626434aae9378df2825aecf" member_type: VOTER last_known_addr { host: "127.8.18.253" port: 46623 } } peers { permanent_uuid: "1c2c2aa8e1d14fa08577c0ddee655983" member_type: VOTER last_known_addr { host: "127.8.18.252" port: 45977 } } }
I20260810 11:36:37.391252 31991 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c2c2aa8e1d14fa08577c0ddee655983 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 11:36:37.410223 32035 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 11:36:37.412341 32035 catalog_manager.cc:1504] Loading token signing keys...
I20260810 11:36:37.439028 32035 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 72340a6f145b46d28372cd7526703a45: Generated new TSK 0
I20260810 11:36:37.440284 32035 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260810 11:36:37.602634 31989 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:36:37.585520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57988 (local address 127.8.18.253:46623)
0810 11:36:37.585748 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0810 11:36:37.585755 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:36:37.599466 (+ 13711us) server_negotiation.cc:407] Connection header received
0810 11:36:37.600590 (+ 1124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:36:37.600597 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:36:37.600746 (+ 149us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:36:37.600837 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:36:37.602336 (+ 1499us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57988: BlockingRecv error: recv got EOF from 127.0.0.1:57988 (error 108)
Metrics: {"server-negotiator.queue_time_us":92}
W20260810 11:36:37.604228 31987 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:36:37.594162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51422 (local address 127.8.18.254:40371)
0810 11:36:37.594424 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0810 11:36:37.594432 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:36:37.602058 (+ 7626us) server_negotiation.cc:407] Connection header received
0810 11:36:37.602341 (+ 283us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:36:37.602348 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:36:37.602438 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:36:37.602516 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:36:37.603867 (+ 1351us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51422: BlockingRecv error: recv got EOF from 127.0.0.1:51422 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
W20260810 11:36:37.606626 31984 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:36:37.598148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38726 (local address 127.8.18.252:45977)
0810 11:36:37.598357 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0810 11:36:37.598364 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:36:37.603993 (+ 5629us) server_negotiation.cc:407] Connection header received
0810 11:36:37.605085 (+ 1092us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:36:37.605091 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:36:37.605189 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:36:37.605269 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:36:37.606310 (+ 1041us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38726: BlockingRecv error: recv got EOF from 127.0.0.1:38726 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
W20260810 11:36:37.648196 31989 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:36:37.639371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58004 (local address 127.8.18.253:46623)
0810 11:36:37.639577 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0810 11:36:37.639584 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:36:37.646098 (+ 6514us) server_negotiation.cc:407] Connection header received
0810 11:36:37.646496 (+ 398us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:36:37.646503 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:36:37.646657 (+ 154us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:36:37.646744 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:36:37.647928 (+ 1184us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58004: BlockingRecv error: recv got EOF from 127.0.0.1:58004 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
W20260810 11:36:37.684952 31989 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:36:37.682343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58020 (local address 127.8.18.253:46623)
0810 11:36:37.682641 (+ 298us) server_negotiation.cc:206] Beginning negotiation
0810 11:36:37.682651 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 11:36:37.683299 (+ 648us) server_negotiation.cc:407] Connection header received
0810 11:36:37.683551 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:36:37.683562 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:36:37.683642 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:36:37.683782 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:36:37.684638 (+ 856us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58020: BlockingRecv error: recv got EOF from 127.0.0.1:58020 (error 108)
Metrics: {"server-negotiator.queue_time_us":89}
W20260810 11:36:37.693377 31989 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:36:37.690508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58036 (local address 127.8.18.253:46623)
0810 11:36:37.691028 (+ 520us) server_negotiation.cc:206] Beginning negotiation
0810 11:36:37.691036 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:36:37.691690 (+ 654us) server_negotiation.cc:407] Connection header received
0810 11:36:37.691917 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:36:37.691924 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:36:37.692019 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:36:37.692127 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:36:37.693109 (+ 982us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58036: BlockingRecv error: recv got EOF from 127.0.0.1:58036 (error 108)
Metrics: {"server-negotiator.queue_time_us":349}
W20260810 11:36:37.731746 31989 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:36:37.726720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58042 (local address 127.8.18.253:46623)
0810 11:36:37.727039 (+ 319us) server_negotiation.cc:206] Beginning negotiation
0810 11:36:37.727049 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 11:36:37.730119 (+ 3070us) server_negotiation.cc:407] Connection header received
0810 11:36:37.730381 (+ 262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:36:37.730386 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:36:37.730463 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:36:37.730584 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:36:37.731476 (+ 892us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58042: BlockingRecv error: recv got EOF from 127.0.0.1:58042 (error 108)
Metrics: {"server-negotiator.queue_time_us":149}
W20260810 11:36:37.746345 31989 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:36:37.743774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58046 (local address 127.8.18.253:46623)
0810 11:36:37.744044 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0810 11:36:37.744053 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 11:36:37.744691 (+ 638us) server_negotiation.cc:407] Connection header received
0810 11:36:37.744951 (+ 260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:36:37.744956 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:36:37.745055 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:36:37.745176 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:36:37.746083 (+ 907us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58046: BlockingRecv error: recv got EOF from 127.0.0.1:58046 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
W20260810 11:36:37.769596 31989 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:36:37.766763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58052 (local address 127.8.18.253:46623)
0810 11:36:37.767064 (+ 301us) server_negotiation.cc:206] Beginning negotiation
0810 11:36:37.767074 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 11:36:37.767765 (+ 691us) server_negotiation.cc:407] Connection header received
0810 11:36:37.768039 (+ 274us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:36:37.768045 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:36:37.768137 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:36:37.768266 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:36:37.769199 (+ 933us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58052: BlockingRecv error: recv got EOF from 127.0.0.1:58052 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
W20260810 11:36:37.807317 31989 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:36:37.805224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58064 (local address 127.8.18.253:46623)
0810 11:36:37.805463 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0810 11:36:37.805470 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 11:36:37.805797 (+ 327us) server_negotiation.cc:407] Connection header received
0810 11:36:37.805978 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:36:37.805986 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:36:37.806110 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:36:37.806249 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:36:37.807050 (+ 801us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58064: BlockingRecv error: recv got EOF from 127.0.0.1:58064 (error 108)
Metrics: {"server-negotiator.queue_time_us":87}
W20260810 11:36:37.876884 31989 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:36:37.874973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58076 (local address 127.8.18.253:46623)
0810 11:36:37.875203 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0810 11:36:37.875208 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:36:37.875649 (+ 441us) server_negotiation.cc:407] Connection header received
0810 11:36:37.875755 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:36:37.875760 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:36:37.875863 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:36:37.875965 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:36:37.876694 (+ 729us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58076: BlockingRecv error: recv got EOF from 127.0.0.1:58076 (error 108)
Metrics: {"server-negotiator.queue_time_us":98}
W20260810 11:36:38.010582 31989 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 11:36:38.008474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58090 (local address 127.8.18.253:46623)
0810 11:36:38.008780 (+ 306us) server_negotiation.cc:206] Beginning negotiation
0810 11:36:38.008788 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 11:36:38.009183 (+ 395us) server_negotiation.cc:407] Connection header received
0810 11:36:38.009295 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:36:38.009299 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:36:38.009380 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:36:38.009532 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0810 11:36:38.010345 (+ 813us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58090: BlockingRecv error: recv got EOF from 127.0.0.1:58090 (error 108)
Metrics: {"server-negotiator.queue_time_us":87}
I20260810 11:36:38.011852 32002 catalog_manager.cc:1259] Loaded cluster ID: ad4e5048dc7a4eeea9efe2407bf6adbb
I20260810 11:36:38.011989 32002 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 1c2c2aa8e1d14fa08577c0ddee655983: loading cluster ID for follower catalog manager: success
I20260810 11:36:38.014781 32002 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 1c2c2aa8e1d14fa08577c0ddee655983: acquiring CA information for follower catalog manager: success
I20260810 11:36:38.016824 32002 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 1c2c2aa8e1d14fa08577c0ddee655983: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 11:36:38.079916 32016 catalog_manager.cc:1259] Loaded cluster ID: ad4e5048dc7a4eeea9efe2407bf6adbb
I20260810 11:36:38.080037 32016 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 7a1346fc8626434aae9378df2825aecf: loading cluster ID for follower catalog manager: success
I20260810 11:36:38.082612 32016 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 7a1346fc8626434aae9378df2825aecf: acquiring CA information for follower catalog manager: success
I20260810 11:36:38.084535 32016 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 7a1346fc8626434aae9378df2825aecf: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 11:36:38.276418 31989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:36:38.270324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58100 (local address 127.8.18.253:46623)
0810 11:36:38.270509 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0810 11:36:38.270514 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:36:38.270905 (+ 391us) server_negotiation.cc:407] Connection header received
0810 11:36:38.271088 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:36:38.271092 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:36:38.271289 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:36:38.271405 (+ 116us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 11:36:38.271858 (+ 453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:36:38.272597 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:36:38.274033 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:36:38.274507 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:36:38.274620 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:36:38.275172 (+ 552us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 11:36:38.275729 (+ 557us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 11:36:38.275845 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0810 11:36:38.275986 (+ 141us) server_negotiation.cc:299] Negotiation successful
0810 11:36:38.276061 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260810 11:36:38.289073 32120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:36:38.282570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38738 (local address 127.8.18.252:45977)
0810 11:36:38.283272 (+ 702us) server_negotiation.cc:206] Beginning negotiation
0810 11:36:38.283277 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 11:36:38.283295 (+ 18us) server_negotiation.cc:407] Connection header received
0810 11:36:38.283395 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:36:38.283399 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:36:38.283554 (+ 155us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:36:38.283714 (+ 160us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 11:36:38.284257 (+ 543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:36:38.284978 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:36:38.286500 (+ 1522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:36:38.286907 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:36:38.287017 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:36:38.287643 (+ 626us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 11:36:38.288283 (+ 640us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 11:36:38.288430 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0810 11:36:38.288583 (+ 153us) server_negotiation.cc:299] Negotiation successful
0810 11:36:38.288674 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":587,"thread_start_us":473,"threads_started":1}
I20260810 11:36:38.308449 31989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:36:38.296294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58108 (local address 127.8.18.253:46623)
0810 11:36:38.296524 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0810 11:36:38.296533 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 11:36:38.297962 (+ 1429us) server_negotiation.cc:407] Connection header received
0810 11:36:38.298552 (+ 590us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:36:38.298562 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:36:38.298696 (+ 134us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:36:38.298783 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 11:36:38.299780 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:36:38.301509 (+ 1729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:36:38.304809 (+ 3300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:36:38.305931 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:36:38.306109 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:36:38.307189 (+ 1080us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 11:36:38.307430 (+ 241us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 11:36:38.307512 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0810 11:36:38.307816 (+ 304us) server_negotiation.cc:299] Negotiation successful
0810 11:36:38.307951 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260810 11:36:38.316952 32120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:36:38.297823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38748 (local address 127.8.18.252:45977)
0810 11:36:38.300223 (+ 2400us) server_negotiation.cc:206] Beginning negotiation
0810 11:36:38.300242 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0810 11:36:38.310580 (+ 10338us) server_negotiation.cc:407] Connection header received
0810 11:36:38.311158 (+ 578us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:36:38.311164 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:36:38.311299 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:36:38.311392 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 11:36:38.312135 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:36:38.312911 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:36:38.314888 (+ 1977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:36:38.315327 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:36:38.315490 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:36:38.315937 (+ 447us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 11:36:38.316053 (+ 116us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 11:36:38.316161 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0810 11:36:38.316381 (+ 220us) server_negotiation.cc:299] Negotiation successful
0810 11:36:38.316524 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2209}
I20260810 11:36:38.317273 32126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 11:36:38.296255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51424 (local address 127.8.18.254:40371)
0810 11:36:38.296967 (+ 712us) server_negotiation.cc:206] Beginning negotiation
0810 11:36:38.296973 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 11:36:38.309700 (+ 12727us) server_negotiation.cc:407] Connection header received
0810 11:36:38.310158 (+ 458us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 11:36:38.310167 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 11:36:38.310429 (+ 262us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 11:36:38.310593 (+ 164us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 11:36:38.311593 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:36:38.312771 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:36:38.314841 (+ 2070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 11:36:38.315262 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 11:36:38.315433 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 11:36:38.315958 (+ 525us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 11:36:38.316457 (+ 499us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 11:36:38.316590 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0810 11:36:38.316832 (+ 242us) server_negotiation.cc:299] Negotiation successful
0810 11:36:38.316945 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":541,"thread_start_us":427,"threads_started":1}
I20260810 11:36:38.323660 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 31754
I20260810 11:36:38.324788 31821 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:36:38.439177 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 31754
W20260810 11:36:38.462803 31928 connection.cc:535] server connection from 127.0.0.1:38704 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260810 11:36:38.463249 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 31833
I20260810 11:36:38.464520 31896 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:36:38.582759 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 31833
I20260810 11:36:38.606413 8267 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task_r3aKW/build/asan/bin/kudu pid 31910
I20260810 11:36:38.607697 31976 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 11:36:38.741707 8267 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task_r3aKW/build/asan/bin/kudu with pid 31910
2026-08-10T11:36:38Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (5040 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (9891 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (396680 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST